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Preface 


This IBM® Redpaper™ publication is a comprehensive guide that covers the IBM Power 
System H922 (9223-22S), and IBM Power System H924 (9223-42S) servers that support 
memory-intensive workloads, such as SAP HANA, and deliver superior price and 
performance for mission-critical applications in IBM AIX®, IBM i, and Linux® operating 
systems. The goal of this paper is to provide a hardware architecture analysis and highlight 
the changes, new technologies, and major features that are being introduced in these 
systems’ 2020 release, such as the following examples: 


> Availability of new IBM POWER9™ processor configurations for the number of cores per 
socket. 


> More performance by using industry-leading IBM Peripheral Component Interconnect® 
Express (PCle) Gené4 slots. 


> Enhanced internal disk configuration options, with up to 14 NVMe adapters (four U.2 
NVMe plus up to 10 PCle add-in cards). 


> Twice as fast back-end I/O enables seamless maximum speed and throughput between 
on-premises and multiple public cloud infrastructures with high availability (HA). 


This publication is for professionals who want to acquire a better understanding of IBM Power 
Systems products. The intended audience includes the following roles: 


Clients 

Sales and marketing professionals 
Technical support professionals 

IBM Business Partners 

Independent software vendors (ISVs) 


vvvvy 


This paper expands the current set of IBM Power Systems documentation by providing a 
desktop reference that offers a detailed technical description of the Power H922 and 
Power H924 systems. 
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General description 


The newest generation of IBM Power System H922 (9223-22S) and IBM Power System H924 
(9223-42S) servers are now updated with new Peripheral Component Interconnect Express 
(PCle) Gen 4 I/O switches. They also include more NVMe adapters support to meet the 
requirements for high-performance and low latency demands that are present in SAP HANA 
environments. 


Optimized for in-memory workloads, these systems deliver increased performance as 
compared to the previous generation, while delivering virtualization and unmatched flexibility 
to help clients run multiple SAP HANA production workloads concurrently, including 
production and non-production such as development and test in a single system. | 


n addition, IBM Power Systems for SAP HANA allow concurrently running virtualized SAP 
HANA side-by-side with other Linux, AIX, or IBM i workloads. It is an infrastructure that 
supports SAP mission-critical workloads but it also features the capabilities that are required 
to support new workloads of the future. 


In this chapter, we discuss IBM Power System H922 (9223-22S) and IBM Power System 
H924 (9223-42S) servers and their features. 


This chapter includes the following topics: 


> 1.1, “Systems overview” on page 2 

> 1.2, “Operating environment” on page 6 

> 1.3, “Physical package” on page 7 

> 1.4, “System features” on page 9 

> 1.5, “Disk and media features” on page 16 

> 1.6, “I/O drawers for Power H922, and Power H924 servers” on page 26 
> 1.7, “System racks” on page 30 

> 1.8, “Hardware Management Console” on page 40 
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1.1 Systems overview 


The following sections provide more information about the Power System H922 for SAP 
HANA (9223-22S) and Power System H924 for SAP HANA (9223-42S) servers. These 
servers are offered to run SAP HANA. They are powered by PCle Gen4 switches, and deliver 
a seamless and lightspeed throughput I/O between multiple on-premises and public clouds 
applications, which make them an ideal platform for in-memory and data-centric applications. 


With this highly flexible system with best-in-class virtualization, clients can consolidate their 
workloads and reduce the footprint in their data center. In addition, clients can run mixed 
workloads with traditional SAP and non-productive workloads that are running in a shared 
pool side by side with production SAP HANA, which leads to an efficient use of the system 
and a high utilization. 


SAP HANA Tailored Datacenter Integration Phase 5 (TDI5) gives clients the flexibility to use 
the storage and networking of their choice, and change configurations, with support from 
SAP, all while achieving a lower total cost of ownership (TCO). 


These servers easily integrate into cloud and cognitive strategies and deliver superior 
on-premises infrastructure for a hybrid cloud in the following ways: 


> Cloud-enabled servers can be used to build an agile, containerized cloud on a server 
platform that is optimized for data and cognitive services. 


> Twice as fast back-end I/O enables seamless maximum speed and throughput between 
on-premises and multiple public cloud infrastructures with high availability (HA). 


> Largest memory bandwidth and memory storage in the market together with numerous 
NVMe adapters that allows a huge virtual machine (VM), container, and bare metal 
consolidation, which saves data center space and networking costs. 


> PowerVM hypervisor is built in at no extra charge; therefore, every POWERS workload is 
virtualized, mobile, and cloud ready. 


> Increased performance and flexibility for your key workloads, including Red Hat Enterprise 
Linux and Red Hat OpenShift deployments. 


> Increased performance and reduced footprint to run IBM Cloud® Paks. 


Important: Consider the following points: 
> All cores must be activated. 


> Linux is required as the primary operating system with a minimum of 75% of the total 
cores. 


> IBM AIX and IBM ican be chosen as secondary operating systems, with a maximum of 
25% of total cores activated across these two operating systems. 


> If the AIX and IBM i total core activation requirements exceed 25%, the Power System 
$924 and Power System S922 models are the preferred servers. 
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1.1.1 Power H922 server (9223-22S) 


The Power H922 (9223-22S) server is a powerful one or two-socket server that includes up to 
22 activated cores. If only one socket is populated at the time of ordering, the second can be 
populated later. 


The server has the I/O configuration flexibility to meet today’s growth and tomorrow’s 
processing needs. This server supports one processor socket option with the following 
offerings: 


4 cores at 2.8 - 3.8 GHz or up to two processor sockets 
8 or 16 cores at 3.4 - 3.9 GHz 

10 or 20 cores at 2.9 - 3.8 GHz 

11 or 22 cores at 2.8- 3.8 GHz 


vvvy 


The systems are in a 19-inch, rack-mount, 2U (EIA units) drawer configuration. All of the 
cores are active. 


The Power H922 (9223-22S) server supports a maximum of 32 DDR4 DIMM slots. If only one 
processor socket is populated, only 16 DIMMs can be used. The following memory features 
are supported: 


16 GB 
32 GB 
64 GB 
128 GB 


vvvy 


They run at speeds of 2133, 2400, and 2666 Mbps, which allows for a maximum system 

memory of 2 TB if one socket is populated and 4 TB with both sockets populated. 

The following I/O options are available in this server that are powered by PCI Gen4 switches: 

> Three x16 Gen4 low-profile, half-length slots (CAPI) 

> Two x16 Gen4 low-profile, half-length slots 

> One x8 Gend4 low-profile, half-length slot (with x16 connectors) (CAPI) 

> One x8 Gend4 low-profile, half-length slot (with x16 connectors) 

> Four x8 Gené4 low-profile, half-length slots (One of these slots is used for the required 
base LAN adapter.) 

The following features are available for the storage backplane: 


#EJ1F: Eight SFF-3s with an optional split card (#EJ1H) 

#EJ1V: Storage backplane with two front PCle Gen4 capable NVMe U.2 drive slots 
#EJ1W: Storage backplane with four front PCle Gen4 capable NVMe U.2 drive slots 
#EJ1G: Expanded function storage backplane 8 SFF-3 bays/single IOA with write cache 


vvvy 


The NVMe options offer fast boot times and is ideally suited to contain the rootvg of Virtual I/O 
Server (VIOS) partitions and it can be used as a way to accelerate SAP HANA start. 
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Figure 1-1 shows a Power H922 server. 


Figure 1-1 Power H922 server 


1.1.2 Power H924 server (9223-42S) 


The Power H924 (9223-42S) server is a powerful one- or two-socket server that includes up 
to 24 activated cores. It has the I/O configuration flexibility to meet today’s growth and 
tomorrow’s processing needs. 


This server supports two processor sockets, and includes the following offerings: 


8 or 16 cores at 3.8 - 4.0 GHz 
10 or 20 cores at 3.5 - 3.9 GHz 
11 or 22 cores at 3.45 - 3.9 GHz 
12 or 24 cores at 3.4 - 3.9 GHz 


vvvy 


The systems are in a 19-inch, rack-mount, 4U (EIA units) drawer configuration. All of the 
cores are active. 


The Power H924 (9223-42S) server supports a maximum of 32 DDR4 DIMM slots. The 
following memory features are supported: 


16 GB 
32 GB 
64 GB 
128 GB 


vvvy 


These offerings allow for a maximum system memory of 4 TB. 


The following features are available for the storage backplane: 

> #EJ1C: 12 SFF-3 bays with am optional split card (#EJ1E) 

> #EJ1D: 18 SFF-3 Bays/Dual IOA with Write Cache 

> #EJ1M: 12 SFF-3 Bays/RDX Bays: 
— Split feature to 6+6 small form factor (SFF) Bays; add a second SAS controller. 
— 12 SFF-3 Bays/RDX Bay/2 EXT PT. 

> #EJ1S: Storage backplane with 6 SFF-3 bays and 2 PCle Gen4 capable NVMe U.2 

>» #EJ1T: Storage backplane with 2 PCle Gen4 capable NVMe U.2 drive slots 

> #EJ1U: Storage backplane with 4 PCle Gen4 capable NVMe U.2 drive slots 
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Each of the backplane options with SFF-3 bays uses leading-edge, integrated SAS RAID 
controller technology that is designed and patented by IBM. 


Figure 1-2 shows the Power H924 (9223-42S) server. 


Figure 1-2. Power Server H924 (9223-42S) 


1.1.3 Common features 


Many features are common to all of the servers that are described in this paper, as described 
next. 


No internal DVD 

An internal DVD option is not available. An external USB DVD drive is available from IBM 
(#EUAS5). Customers are encouraged to use USB flash drives to install operating systems and 
VIOS whenever possible because they are much faster than a DVD. 


Operator panel 
The operator panel is formed by two parts. All of the servers include the first part, which 
provides the power switch and LEDs, as shown in Figure 1-3. 


Figure 1-3 Operator panel: Power switch and LEDs 


The second part is an LCD panel with three buttons, as shown in Figure 1-4. 


Figure 1-4 Operator panel: LCD and switches 
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The LCD panel is optional in the Power H922 server and Power H924 rack-mounted servers; 
however, if a rack contains any of the IBM POWER39 processor-based scale-out servers, one 
of the them must have an LCD panel. 


The LCD panel can be moved (by using the correct procedure) from one server to another so 
that the appropriate service can be completed. 


1.2 Operating environment 


Table 1-1 lists the electrical characteristics of the servers. 


Table 1-1 Electrical characteristics of Power H922 and Power H924 servers 


Electrical characteristics Properties 


Power H922 server Power H924 server 


Operating voltage 1400 W power supply: 1400 W power supply: 
200 - 240 V AC 200 - 240 V AC 


Operating frequency 47/63 Hz 47/63 Hz 


Thermal output 6,416 Btu/hour (maximum) 9,386 Btu/hour (maximum) 
1880 watts (maximum) 2750 watts (maximum) 


Power-source loading 1.94 kVa (maximum 2.835 kVa (maximum 
= a 


Note: The maximum measured value is the worst-case power consumption that is 
expected from a fully populated server under an intensive workload. The maximum 
measured value also accounts for component tolerance and non-ideal operating 
conditions. Power consumption and heat load vary greatly by server configuration and 
utilization. 


For more information about a heat output estimate that is based on a specific 
configuration, see the IBM System Energy Estimator. 


Table 1-2 lists the environment requirements for the servers. 


Table 1-2 Environment requirements for Power H922 and Power H924 servers 


Environment Recommended Allowable operating Non-operating 
operating 

Temperature 18 - 27°C 5 - 40°C (41 - 104°F) 5 - 45°C (41 - 113°F) 
(64.4 - 80.6°F) 


Humidity range 5.5°C (42°F) dew point | 8% - 85% RH 8% - 80% RH 
(DP) to 60% relative 
humidity (RH) or 15°C 
(59°F) dew point 


Maximum dew point 24°C (75°F) 27°C (80°F) 
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Environment Recommended Allowable operating Non-operating 
operating 


Maximum operating N/A 3050 m (10000 ft) N/A 
altitude 


Table 1-3 lists the noise emissions for the systems. 


Table 1-3 Noise emissions for Power H922 and H924 


Declared A-weighted sound power Declared A-weighted sound 
level, Lwag (B) pressure level, Ly am (dB) 


[opeing [te | Ovwatne [te 


Power H922 7.8 61 53 
server 
Power H924 6.4 5.2 34 
server 


Tip: Consider the following points: 


> Declared level Lywag is the upper-limit A-weighted sound power level. Declared level 
Loam Is the mean A-weighted emission sound pressure level that is measured at the 
1-meter bystander positions. 


> All measurements are made in conformance with ISO 7779 and declared in 
conformance with ISO 9296. 


> 10 dB (decibel) equals 1 B (bel). 


1.3 Physical package 


Table 1-4 lists the physical dimensions of the Power H922 chassis. The server is available in a 
rack-mounted form factor and takes 2U (2 EIA units) of rack space. 


Table 1-4 Physical dimensions of the rack-mounted Power H922 chassis 
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Figure 1-5 show the front view of the Power H922 system. 


Figure 1-5 Front view of the Power H922 system 


Table 1-5 lists the physical dimensions of the rack-mount Power H924 chassis. The server is 
available in a rack-mounted form factor and takes 4U (4 EIA units) of rack space. 


Table 1-5 Physical dimensions of the rack-mount Power H924 chassis 


Figure 1-6 shows the front view of the Power H924 system. 


Figure 1-6 Front view of the Power H924 system 
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1.4 System features 


The system chassis contains up to two processor modules. Each of the POWER processor 
chips in the server includes a 64-bit architecture, up to 512 KB of L2 cache per core, and up 
to 10 MB of L3 cache per core. All of the cores are active. 


1.4.1 Power H922 (9223-22S) system features 


The Power H922 (9223-22S) server includes the following standard features: 


> 


POWERS processor modules: 


4-core typical 2.8 - 3.8 GHz (maximum) POWER9 processor 
— 8-core typical 3.4 - 3.9 GHz (maximum) POWER39 processor 
10-core typical 2.9 - 3.8 GHz (maximum) POWER9 processor 
— 11-core typical 2.8 - 3.8 GHz (maximum) POWER9 processor 


High-performance Mbps DDR4 ECC memory: 


— 16 GB, 32 GB, 64 GB, or 128 GB memory features. Different sizes and configurations 
run at different frequencies of 2133, 2400, and 2666 Mbps. 


— Upto 4 TB of DDR4 memory with two POWER® processors. 

— Up to 2 TB of DDR4 memory with one POWER processor. 
Storage feature: 

— Storage Backplane Gen 4 with two or four NVMe U.2 drive slots. 


— Eight SFF bays, one integrated SAS controller without cache, and JBOD, RAID 0, 
RAID 5, RAID 6, or RAID 10. Optionally, split the SFF-3 bays and add a second 
integrated SAS controller without cache. 


— Expanded Function Storage Backplane 8 SFF-3 bays/Single IOA with Write Cache. 
Optionally, attach an EXP12SX or EXP24SX SAS hard disk drive (HDD) or solid-state 
drive (SSD) Expansion Drawer to the single IOA. 


One PCle3 NVMe carrier card can be ordered only with a storage backplane. If a PCle3 
NVMe carrier card is ordered with a storage backplane, the optional split feature is not 
supported. 


PCle slots with a single processor: 


— Three x16 Gen 4 low-profile (LP), half-length slots (one of these slots is Coherent 
Accelerator Processor Interface [CAPI] capable). 


— One x8 Gen 4 LP, half-length slot with x16 connector (CAPI-capable). 


— Four x8 Gen 4 LP, half-length slots (one of these slots is used for the required base 
local area network [LAN] adapter). 


PCle slots with two processors: 
— Five x16 Gen 4 LP, half-length slots (three of these slots are CAPI-capable). 


— Two x8 Gen 4 LP, half-length slots with x16 connectors (one of these slots is 
CAPI-capable). 


— Four x8 Gen 4 LP, half-length slots (one of these slots is used for the required base 
LAN adapter). 


Integrated features: 


— Service processor 
— EnergyScale technology 


Chapter 1. General description 9 


— Hot-plug and redundant cooling 

— Two front USB 3.0 ports 

— Two rear USB 3.0 ports 

— Two Hardware Management Console (HMC) 1 GbE RJ45 ports 
— One system port with an RJ45 connector 

— Two hot-plug, redundant power supplies 

— 19-inch rack-mounting hardware (2U) 


1.4.2 Power H924 (9223-42S) system features 


The Power H924 servers include the following standard features: 


> 


POWERS processor-based modules: 


8-core typical 3.8 - 4.0 GHz (maximum) POWER39 processor. 
10-core typical 3.5 - 3.9 GHz (maximum) POWER39 processor. 
11-core typical 3.45 - 3.9 GHz (maximum) POWER39 processor. 
12-core typical 3.4 - 3.9 GHz (maximum) POWER39 processor. 


High-performance Mbps DDR4 ECC memory: 


— 16 GB, 32 GB, 64 GB, or 128 GB memory. Different sizes and configurations run at 
different frequencies of 2133, 2400, and 2666 Mbps. 


— Upto 4 TB of DDR4 memory with two POWER processors. 
— Up to 2 TB of DDR4 memory with one POWER processor. 
Storage feature: 


— Base 12 SFF-3 bays/RDX bay. Optionally, split the SFF-3 bays and add a second 
integrated SAS controller without cache. 


— Expanded function 18 SFF-3 bays/Dual IOA with Write Cache and Optional External 
SAS port. 


— Expanded function 12 SFF-3 bays/RDX Bay/Dual IOA with Write Cache and Optional 
External SAS port. Optionally, attach an EXP12SX or EXP24SX SAS HDD/SSD 
Expansion Drawer to the dual IOA. 


— #EJ1S: Gen 4 Backplane with six SFF-3 bays and two NVMe U.2 drives. 
— #EJ1T: Two NVMe U.2 drive slots. 

— #EJ1U: Four NVMe U.2 drive slots. 

PCle slots with a single processor: 


— Three x16 Gen 4 full height half length (FHHL) slots (one of these slots is 
CAPI-capable). 


— One x8 Gen 4 FHHL slot with x16 connector (CAPI-capable). 


— Four x8 Gen 4 FHHL slots (one of these slots is used for the required base LAN 
adapter). 


PCle slots with two processors: 
— Five x16 Gen 4 FHHL slots (three of these slots are CAPI-capable). 
— Two x8 Gen 4 FHHL slots with x16 connectors (one of these slots is CAPI-capable). 


— Four x8 Gen 4 FHHL slots (one of these slots is used for the required base LAN 
adapter). 


Integrated features: 


— Service processor 
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— EnergyScale technology 

— Hot-swap and redundant cooling 

— One front USB 3.0 port 

— Two rear USB 3.0 ports 

— Two HMC 1 GbE RuJ45 ports 

— One system port with an RJ45 connector 
— Four hot-plug, redundant power supplies 
— 19-inch rack-mounting hardware (4U) 


1.4.3 Minimum features 


The minimum Power H922 initial order must include the following components: 


> 


> 


> 


> 


> 


One processor module 
Two 16 GB RDIMMs 
Two power supplies and power cords 
One operating system indicator 
One cover set indicator 
One Language Group Specify 
One of the following storage options and one of the following network options: 
— Storage options: 
¢ For boot from NVMe: One NVMe carrier and one NVMe U.2 Module. 


¢ For boot from a local SFF-3 HDD/SDD: One storage backplane and one SFF-3 
HDD or SDD. 


¢ For boot from SAN: An internal HDD or SSD and RAID card are not required if Boot 
from SAN (#0837) is selected. A Fibre Channel adapter must be ordered if 
#0837 is selected. 


— Network options: 


¢ One PCle2 4-port 1 Gb Ethernet adapter. 
¢ One of the supported 10 Gb Ethernet adapters. 


The minimum Power H9?24 initial order must include the following components: 


> 


> 


> 


One processor module 
Two 16 GB RDIMMs 
Four power supplies and power cords 
One operating system indicator 
One cover set indicator 
One Language Group Specify 
One of the following storage options and one of the following network options: 
— Storage options: 
¢ For boot from NVMe: One NVMe carrier and one NVMe U.2 Module. 


¢ For boot from a local SFF-3 HDD/SDD: One storage backplane and one SFF-3 
HDD or SDD. 


¢ For boot from SAN: An internal HDD or SSD and RAID card are not required if Boot 
from SAN (#0837) is selected. A Fibre Channel adapter must be ordered if 
#0837 is selected. 
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— Network options: 


¢ One PCle2 4-port 1 Gb Ethernet adapter. 
¢ One of the supported 10 Gb Ethernet adapters. 


1.4.4 Power supply features 


The Power H922 server supports two 1400 W 200 - 240 V AC (#EB2M) power supplies. Two 
power supplies are always installed. One power supply is required during the boot phase and 
for normal system operation, and the second is for redundancy. 


The Power H924 server supports four 1400 W 200 - 240 V AC (#EB2M) power supplies. Four 
power supplies are always installed. Two power supplies are required during the boot phase 
and for normal system operation, and the third and fourth ones are for redundancy. 


Note: If one of the redundant power supplies fails, an Early Power Off Warning (EPOW) is 
sent to the service processor and the server runs a shutdown. 


1.4.5 Processor module features 


The following section describes the processor modules that are available for the Power H922 
and Power H924 servers. 


Power H922 processor modules 

A maximum of one processor with four processor cores (#EP76), or a maximum of two 
processors of either eight processor cores (#EP77), 10 processor cores (#EP78), or 11 
processor cores (#EP79) is allowed. All processor cores must be activated. 


The following quantities of processor activation entitlements are available: 


> One 4-core typical 2.8 - 3.8 GHz (maximum) processor (#EP76) requires that four 
processor activation codes be ordered. A maximum of four processor activations (#EP7C) 
is allowed. 


> One 8-core typical 3.4 - 3.9 GHz (maximum) processor (#EP77) requires that eight 
processor activation codes be ordered. A maximum of eight processor activations 
(#EP7D) is allowed. 


> Two 8-core typical 3.4 - 3.9 GHz (maximum) processors (#EP77) require that 16 processor 
activation codes be ordered. A maximum of 16 processor activations (#EP7D) is allowed. 


> One 10-core typical 2.9 - 3.8 GHz (maximum) processor (#EP78) requires that 10 
processor activation codes be ordered. A maximum of 10 processor activation feature 
codes (#EP7E) is allowed. 


> Two 10-core typical 2.9 - 3.8 GHz (maximum) processors (#EP78) require that 20 
processor activation codes be ordered. A maximum of 20 processor activation feature 
codes (#EP7E) is allowed. 


> One 11-core typical 2.8 - 3.8 GHz (maximum) processors (#EP79) require that 11 
processor activation codes be ordered. A maximum of 11 processor activation feature 
codes (#EP7F) is allowed. 


> Two 11-core typical 2.8 - 3.8 GHz (maximum) processors (#EP79) require that 22 
processor activation codes be ordered. A maximum of 22 processor activation feature 
codes (#EP7F) is allowed. 
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Table 1-6 lists the processor features that are available for the Power H922 server. 


Table 1-6 Processor features for the Power H922 server 


Power H924 processor modules 

A maximum of two processors with eight processor cores (#EP7J), two processors with 10 
processor cores (#EP7K), two processors with 11 processor cores (#EP7M), or two 
processors with 12 processor cores (#EP7L) is allowed. All processor cores must be 
activated. 


The following quantities of processor activation entitlements are available: 


> One 8-core typical 3.8 - 4.0 GHz (maximum) processor (#EP7J) requires that eight 
processor activation codes be ordered. A maximum of eight processor activations 
(#EP7P) is allowed. 


> Two 8-core typical 3.8 - 4.0 GHz (maximum) processors (#EP7J) require that 16 processor 
activation codes be ordered. A maximum of 16 processor activations (#EP7P) is allowed. 


> One 10-core typical 3.5 - 3.9 GHz (maximum) processor (#EP7K) requires that 10 
processor activation codes be ordered. A maximum of 10 processor activation feature 
codes (#EP7Q) is allowed. 


> Two 10-core typical 3.5 - 3.9 GHz (maximum) processors (#EP7K) require that 20 
processor activation codes be ordered. A maximum of 20 processor activation feature 
codes (#EP7Q) is allowed. 


> One11-core typical 3.45 - 3.9 GHz (maximum) processors (#EP7M) require that 11 
processor activation codes be ordered. A maximum of 11 processor activation feature 
codes (#EP7S) is allowed. 


> Twott-core typical 3.45 - 3.9 GHz (maximum) processors (#EP7M) require that 22 
processor activation codes be ordered. A maximum of 22 processor activation feature 
codes (#EP7S) is allowed. 


> One12-core typical 3.4 - 3.9 GHz (maximum) processors (#EP7L) require that 12 
processor activation codes be ordered. A maximum of 12 processor activation feature 
codes (#EP7R) is allowed. 


> Two 12-core typical 3.4 - 3.9 GHz (maximum) processors (#EP7L) require that 24 
processor activation codes be ordered. A maximum of 24 processor activation feature 
codes (#EP7R) is allowed. 
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Table 1-7 lists the processor features that are available for the Power H924 server. 


Table 1-7 Processor features for the Power H924 server 


Processor module description 

8-core typical 3.8 - 4.0 GHz (maximum) POWER39 processor 
10-core typical 3.5 - 3.9 GHz (maximum) POWER39 processor 
11-core typical 3.45 - 3.9 GHz (maximum) POWER39 processor 


1.4.6 Memory features 


A minimum of 32 GB of memory is required on the Power H922 and Power H924 systems. 
Memory upgrades require memory pairs. The base memory is four 8-GB DDR4 memory 
modules (#EM6J). 


Table 1-8 lists the memory features that are available for the Power H922 and Power H924 
systems. 


Table 1-8 Summary of memory features for the Power H922 and Power H924 servers 


Note: Different sizes and configurations run at different frequencies of 2133, 2400, and 
2666 Mbps. 


14 IBM Power Systems H922 and H924 Technical Overview and Introduction 


1.4.7 PCle slots 


The following PCle slots are available: 


> The Power H922 server has up to 11 PCle hot-plug slots. The 2U height of the server 
allows only for the installation of PCle adapters in the half-height half-length (HHHL) LP 
form factor. 


— With two POWERS processor single-chip modules (SCMs), 11 PCle slots are 
available: 


Three x16 Gen 4 CAPI-capable slots with each directly connected to an SCM. 


One x8 Gen 4 with x16 connector CAPI-capable slot that is directly connected to an 
SCM. 


One x8 Gen 4 with x16 connector slot that is directly connected to an SCM. 
Two x16 Gen 4 slots that are connected to an internal Gen 4 switch. 
Four x8 Gen 4 slots that are connected to an internal Gen 4 switch. 


— With one POWER9 processor SCM, eight PCle slots are available: 


One x16 Gen 4 CAPI-capable slot that is directly connected to an SCM. 


One x8 Gen 4 with x16 connector CAPI-capable slot that is directly connected to an 
SCM. 


Two x16 Gen 4 slots that are connected to an internal Gen 4 switch. 
Four x8 Gen 4 slots that are connected to an internal Gen 4 switch. 


> The Power H924 server has up to 11 PCle hot-plug slots that support the installation of 
PCle adapters with full-height half-length (FHHL) form factor only: 


— With two POWERS9 processor SCMs, 11 PCle slots are available: 


Three x16 Gen 4 CAPI-capable slots with each directly connected to an SCM. 


One x8 Gen 4 with x16 connector CAPI-capable slot that is directly connected to an 
SCM. 


One x8 Gen 4 with x16 connector slot that is directly connected to an SCM. 
Two x16 Gen 4 slots that are connected to an internal Gen 4 switch. 
Four x8 Gen 4 slots that are connected to an internal Gen 4 switch. 


— With one POWERS processor SCM, eight PCle slots are available: 


One x16 Gen 4 CAPI-capable slot that is directly connected to an SCM. 


One x8 Gen 4 with x16 connector CAPI-capable slot that is directly connected to an 
SCM. 


Two x16 Gen 4 slots that are connected to an internal Gen 4 switch. 
Four x8 Gen 4 slots that are connected to an internal Gen 4 switch. 


The x16 slots can provide up to twice the bandwidth of x8 slots because they offer twice as 
many PCle lanes. PCle Gen 4 slots can support up to twice the bandwidth of a PCle Gen3 
slot, and PCle Gen3 slots can support up to twice the bandwidth of a PCle Gen2 slot, 
assuming an equivalent number of PCle lanes. 
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At least one PCle Ethernet adapter is required on the server by IBM to ensure correct 
manufacturing, testing, and support of the server. One of the x8 PCle slots is used for this 
required adapter. These servers are smarter about energy efficiency when cooling the PCle 
adapter environment. They sense which IBM PCle adapters are installed in their PCle slots 
and, if an adapter requires higher levels of cooling, they automatically speed up fans to 
increase airflow across the PCle adapters. Faster fans increase the sound level of the server. 


1.5 Disk and media features 


The following backplane options are available for the Power H922 (9223-22S) servers: 
> Base Storage Backplane 8 SFF-3 bays (#EJ1F) 
> 44+ 4 SFF-3 bays split backplane (#EJ1H) 


> Expanded function Storage Backplane 8 SFF-3 bays/Single IOA with Write Cache 
(#EJ1G) 


> Storage Backplane Gen 4 with two NVMe U.2 drive slots (#EJ1V) 
> Storage Backplane Gen 4 with four NVMe U.2 drive slots (#EJ1W) 


The Storage Backplane option (#EJ1F) provides eight SFF-3 bays and one SAS controller 
with zero write cache. 


By optionally adding the Split Backplane (#EJ1H), a second integrated SAS controller with no 
write cache is provided, and the eight SSF-3 bays are logically divided into two sets of four 
bays. Each SAS controller independently runs one of the four-bay sets of drives. 

The following backplane options are available for the Power H924 (9223-42S) servers: 

> Base Storage Backplane 12 SFF-3 bays/RDX bay (#EJ1C) 

> 6+6 SFF-3 bays split backplane for #EJ1C (#EJ1E) 


> Expanded function Storage Backplane 18 SFF-3 bays/Dual IOA with Write Cache and 
optional external SAS port (#EJ1D) 


> Expanded function Storage Backplane 12 SFF-3 bays/RDX bay/Dual IOA with Write 
Cache and optional external SAS port (#EJ1M) 


> Storage Backplane Gen 4 with six SFF-3 Bays and two NVMe U.2 drive slots (#EJ1S) 
> Storage Backplane Gen 4 with two NVMe U.2 drive slots (#EJ1T) 
> Storage Backplane Gen 4 with four NVMe U.2 drive slots (#EJ1U) 


The Base Storage Backplane option (#EJ1C) provides 12 SFF-3 bays and one SAS controller 
with zero write cache. 


By optionally adding the Split Backplane (#EJ1E), a second integrated SAS controller with no 
write cache is provided, and the 12 SSF-3 bays are logically divided into two sets of six bays. 
Each SAS controller independently runs one of the six-bay sets of drives. 


The backplane options provide SFF-3 SAS bays in the system unit. These 2.5-inch or SFF 
SAS bays can contain SAS drives (HDD or SSD) mounted on a Gen tray or carrier. Thus, 
the drives are designated SFF-3. SFF-1 or SFF-2 drives do not fit in an SFF-3 bay. All SFF-3 
bays support concurrent maintenance or hot-plug capability. 
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These backplane options use leading-edge, integrated SAS RAID controller technology that 
is designed and patented by IBM. A custom-designed IBM PowerPC® based ASIC chip is the 
basis of these SAS RAID controllers. It provides RAID 5 and RAID 6 performance levels, 
especially for SSDs. Internally, SAS ports are implemented and provide plenty of bandwidth. 
The integrated SAS controllers are placed in dedicated slots and do not reduce the number of 
available PCle slots. 


This backplane option supports HDDs or SSDs or a mixture of HDDs and SSDs in the SFF-3 
bays. Mixing HDDs and SSDs applies even within a single set of six bays of the split 
backplane option. 


Note: If you mix HDDs and SSDs, they must be in separate arrays (unless you use the 
IBM Easy Tier® function). 


This backplane option offers the following drive protection options: 


RAID 0 

RAID 5 (requires a minimum of three drives of the same capacity) 
RAID 6 (requires a minimum of four drives of the same capacity) 
RAID 10 (requires a minimum of two drives) 


vvvy 


Hot-spare capability is supported by RAID 5, RAID 6, and RAID 10. 


This backplane option is supported by AIX and Linux, VIOS, and IBM i. It is highly 
recommended (but not required) that the drives be protected. With IBM i, all drives are 
required to be protected by RAID or mirroring. 


Unlike the hot-plug PCle slots and SAS bays, concurrent maintenance is not available for the 
integrated SAS controllers. Scheduled downtime is required if a service action is required for 
these integrated resources. 


In addition to supporting HDDs and SSDs in the SFF-3 SAS bays, the Expanded Function 
Storage Backplane (#EJ1G) supports the optional attachment of an EXP12SX or EXP24SX 
Expansion Drawer. All bays are accessed by both of the integrated SAS controllers. The bays 
support concurrent maintenance (hot-plug). 


Note: FC EC5X is an important new entry level point for NVMe to boot AIX, Linux, and 
VIOS. 


Table 1-9 lists the available disk drive feature codes that can be installed in the Power H922 
(9223-22S) server. 


Table 1-9 Disk drive feature code description for the Power H922 (9223-22S) server 


1953 1981 | 300 GB 15k RPM SAS SFF-2 Disk Drive (AIX/Linux) AIX and Linux 


ESDB 59EO | 300 GB 15 K RPM SAS SFF-3 Disk Drive AIX and Linux 
(AIX/Linux) 
ESNM 5B43 | 300 GB 15 K RPM SAS SFF-2 4k Block Cached 
Disk Drive (AIX/Linux 


ES90 5B13 | 387 GB Enterprise SAS 4k SFF-3 SSD for AlX/Linux 


AIX and Linux 


AIX, IBM i, and Linux 
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ESNK 5B41 300 GB 15 K RPM SAS SFF-3 4k Block Cached AIX and Linux 
Disk Drive (AIX/Linux) 
- 672 
inux) 


ES94 5B10 | 387 GB Enterprise SAS 4k SFF-2 SSD for AIX/Linux AIX, IBM i, and Linux 


ESBO 5B19 387 GB Enterprise SAS 5xx SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESB2 5B16 387 GB Enterprise SAS 5xx SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


ESB8 5B13 | 387 GB Enterprise SAS 4k SFF-3 SSD for AIX/Linux aa AIX, IBM i, and Linux 
ESBA 5B10 | 387 GB Enterprise SAS 4k SFF-2 SSD for AIX/Linux AIX, IBM i, and Linux 


ESGT 5B19 | 387 GB Enterprise SAS 5xx SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESGV 5B16 | 387 GB Enterprise SAS 5xx SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


1964 19B3 | 600 GB 10k RPM SAS SFF-2 Disk Drive (AIX/Linux) AIX and Linux 
ESDS5 59D0 | 600 GB 10K RPM SAS SFF-3 Disk Drive (AIX/Linux) AIX and Linux 


ESEV 59D2 | 600 GB 10K RPM SAS SFF-2 Disk Drive 4 K Block 
- 4096 

ESF5 59D3 | 600 GB 10K RPM SAS SFF-3 Disk Drive 4 K Block 
- 4096 

ESNP 5B45 | 600 GB 15 K RPM SAS SFF-3 4k Block Cached 
Disk Drive (AIX/Linux) 


AIX and Linux 
AIX and Linux 


AIX and Linux 


AIX, IBM i, and Linux 


ESB4 5B1A_ | 775 GB Enterprise SAS 5xx SFF-3 SSD for 
AIX/Linux 

ESB6 5B17 | 775 GB Enterprise SAS 5xx SFF-2 SSD for 
AIX/Linux 


ESBE 5B14 775 GB Enterprise SAS 4k SFF-3 SSD for AIX/Linux AIX, IBM i, and Linux 
ESBG 5B11_ | 775 GB Enterprise SAS 4k SFF-2 SSD for AIX/Linux AIX, IBM i, and Linux 


ESGX 5B1iA 775 GB Enterprise SAS 5xx SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESGZ 5B17 775 GB Enterprise SAS 5xx SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


ESNA 5B11_ | 775 GB Enterprise SAS 4k SFF-2 SSD for AIX/Linux AIX, IBM i, and Linux 
AIX, IBM i, and Linux 
ESJO 5B29 931 GB Mainstream SAS 4k SFF-2 SSD for : i 
AIX/Linux 


AIX, IBM i, and Linux 

ESJ8 5B2B 931 GB Mainstream SAS 4k SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESJJ 5B29 931 GB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 
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AIX, IBM i, and Linux 


ESNR 5B47 600 GB 15 K RPM SAS SFF-2 4k Block Cached 672 AIX and Linux 
Disk Drive (AIX/Linux) 


ESJS 5B2B | 931 GB Mainstream SAS 4k SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESF3 59DA | 1.2 TB 10K RPM SAS SFF-2 Disk Drive 4 K Block - | 672 AIX and Linux 
4096 

ESF9 59DB | 1.2 TB 10K RPM SAS SFF-83 Disk Drive 4 K Block - AIX and Linux 
4096 

ESBJ 5B15 1.55 TB Enterprise SAS 4k SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESBL 5B12 1.55 TB Enterprise SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 

ESNE 5B12 1.55 TB Enterprise SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 

ESNG 5B15 1.55 TB Enterprise SAS 4k SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

EC5G 58FC | PCle3 x8 LP 1.6 TB NVMe Flash adapter for 5 AIX, IBM i, and Linux 
AIX/Linux 

EC5X 5987 Mainstream 800 GB SSD NVMe U.2 module for 4 AIX, IBM i, and Linux 
AIX/Linux 

ES1E 59B8 1.6 TB Enterprise 4 K NVMe SFF U.2 15 mm SSD 4 AIX, IBM i, and Linux 
PCle4 for AIX/Linux 

ES1F 59B8 1.6 TB Enterprise 4 K NVMe SFF U.215mmSSD | 4 IBM i 
PCle4 for IBM i 

ESFT 59DD | 1.8TB 10K RPM SAS SFF-2 Disk Drive 4 K Block - | 672 AIX and Linux 
4096 

ESFV 59DE | 1.8TB 10K RPM SAS SFF-3 Disk Drive 4 K Block - AIX and Linux 
4096 

ESJ2 5B21 1.86 TB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 

ESJA 5B20 1.86 TB Mainstream SAS 4k SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESJL 5B21 1.86 TB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 

ESJU 5B20 1.86 TB Mainstream SAS 4k SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

EC5C 58FD | PCle3 x8 LP 3.2 TB NVMe Flash adapter for 5 AIX, IBM i, and Linux 
AIX/Linux 

EC7C 594B PCle4 LP 3.2 TB NVMe Flash adapter x8 for 5 AIX, IBM i, and Linux 
AIX/Linux 

ES1G 59B9 3.2 TB Enterprise 4 K NVMe SFF U.2 15 mm SSD 4 AIX, IBM i, and Linux 
PCle4 for AIX/Linux 

ESJ4 5B2D | 3.72 TB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 
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ESJC 5B2C | 3.72 TB Mainstream SAS 4k SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESJN 5B2D | 3.72 TB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 

ESJW 5B2C | 3.72 TB Mainstream SAS 4k SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ES62 5B1D | 3.86 - 4.0 TB 7200 RPM 4 K SAS LFF-1 Nearline 336 AIX and Linux 
Disk Drive (AIX/Linux) 

EC5E 58FE PCle3 x8 LP 6.4 TB NVMe Flash adapter for 5 AIX, IBM i, and Linux 
AIX/Linux 

EC7E 594C_ | PCle4 LP 6.4 TB NVMe Flash adapter x8 for 5 AIX, IBM i, and Linux 
AIX/Linux 

EC5V 6.4 TB Enterprise 4 K NVMe SFF U.2 15 mm SSD 4 AIX, IBM i, and Linux 
PCle4 for AIX/Linux 

ESJE 5B2E 7.45 TB Mainstream SAS 4k SFF-3 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESJ6 5B2F 7.45 TB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 

ES64 5B1F 7.72 - 8.0 TB 7200 RPM 4 K SAS LFF-1 Nearline 336 AIX and Linux 
Disk Drive (AIX/Linux) 


EC CC 
19B1 | Quantity 150 of #1953 a en 
5B43 | Quantity 150 of #ESNM (300 GB 15k SFF-2) 
5B47 | Quantity 150 of #ESNR (600 GB 15k SFF-2) 4 


EQ62 5B1D | Quantity 150 of #ES62 3.86 - 4.0 TB 7200 rpm 4k 2 
LFF-1 Disk 
EQ64 5B1F 


59D2 | Quantity 150 of #ESEV (600 GB 10k SFF-2) 
59DA | Quantity 150 of #ESF3 (1.2 TB 10k SFF-2) 
59DD | Quantity 150 of #ESFT (1.8 TB 10k SFF-2) 
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Quantity 150 of #ES64 7.72 - 8.0 TB 7200 rpm 4k 2 
LFF-1 Disk 


Quantity 150 of #ESJO 931 GB SAS 4k 
Quantity 150 of #ESJ2 1.86 TB SAS 4k 
Quantity 150 of #ESJ4 3.72 TB SAS 4k 
Quantity 150 of #ESJ6 7.45 TB SAS 4k 
Quantity 150 of #ESB2 387 GB SAS 4k 
Quantity 150 of #ESB6 775 GB SAS 4k 
Quantity 150 of #ESBA 387 GB SAS 4k 
Quantity 150 of #ESBG 775 GB SAS 4k 


ESQL 5B12 | Quantity 150 of #ESBL 1.55 TB SAS 4k Pa aaa 


Table 1-10 lists the available disk drive feature codes that can be installed in the Power H924 
(9223-42S) server. 


Table 1-10 Disk drive feature code description for the Power H924 (9223-42S) server 


ESBO 5B19 387 GB Enterprise SAS 5xx SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 

ESB2 5B16 387 GB Enterprise SAS 5xx SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 

ESB4 5B1A_ | 775 GB Enterprise SAS 5xx SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 

ESB6 5B17 775 GB Enterprise SAS 5xx SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


ESB8 5B13 | 387 GB Enterprise SAS 4k SFF-3 SSD for AIX/Linux AIX, IBM i, and Linux 


ESB9 5B13 | 387 GB Enterprise SAS 4k SFF-3 SSD for IBM i AIX, IBM i, and Linux 


ESBJ 5B15 1.55 TB Enterprise SAS 4k SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 


ESBK 5B15 | 1.55 TB Enterprise SAS 4k SFF-3 SSD for IBM i AIX, IBM i, and Linux 


ESBL 5B12 1.55 TB Enterprise SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


ESBM 5B12 | 1.55 TB Enterprise SAS 4k SFF-2 SSD for IBM i AIX, IBM i, and Linux 


ESJO 5B29 931 GB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


ESJ1 5B29 931 GB Mainstream SAS 4k SFF-2 SSD for IBM i AIX, IBM i, and Linux 


ESJ2 5B21 1.86 TB Mainstream SAS 4k SFF-2 SSD for AIX, IBM i, and Linux 
AIX/Linux 

ESJ4 5B2D | 3.72 TB Mainstream SAS 4k SFF-2 SSD for 
AIX/Linux 


ESJ5 5B2D | 3.72 TB Mainstream SAS 4k SFF-2 SSD for IBM i 


AIX, IBM i, and Linux 


AIX, IBM i, and Linux 


ESJ3 5B21 | 1.86 TB Mainstream SAS 4k SFF-2 SSD for IBM i AIX, IBM i, and Linux 
| 
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ESJ6 5B2F 7.45 TB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


ESJ7 5B2F | 7.45 TB Mainstream SAS 4k SFF-2 SSD for IBM i AIX, IBM i, and Linux 


ESJ8 5B2B 931 GB Mainstream SAS 4k SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 


ESJ9 5B2B_ | 931 GB Mainstream SAS 4k SFF-3 SSD for IBM i AIX, IBM i, and Linux 


ESJA 5B20 1.86 TB Mainstream SAS 4k SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 


ESJB 5B20 | 1.86 TB Mainstream SAS 4k SEF-3 SSD for IBM i AIX, IBM i, and Linux 


ESJC 5B2C | 3.72 TB Mainstream SAS 4k SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 


ESJD 5B2C_ | 3.72 TB Mainstream SAS 4k SEF-3 SSD for IBM i AIX, IBM i, and Linux 


ESJE 5B2E 7.45 TB Mainstream SAS 4k SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 


ESJF 5B2E | 7.45 TB Mainstream SAS 4k SEF-3 SSD for IBM i AIX, IBM i, and Linux 


ESJJ 5B29 931 GB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


ESJK 5B29 | 931 GB Mainstream SAS 4k SFF-2 SSD for IBM i AIX, IBM i, and Linux 


ESJL 5B21 1.86 TB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


ESJM 5B21 | 1.86 TB Mainstream SAS 4k SFF-2 SSD for IBM i AIX, IBM i, and Linux 


ESJN 5B2D | 3.72 TB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


ESJP 5B2D | 3.72 TB Mainstream SAS 4k SFF-2 SSD for IBM i AIX, IBM i, and Linux 


ESJQ 5B2F 7.44 TB Mainstream SAS 4k SFF-2 SSD for 336 AIX, IBM i, and Linux 
AIX/Linux 


5B2F | 7.44 TB Mainstream SAS 4k SFF-2 SSD for IBM i AIX, IBM i, and Linux 


931 GB Mainstream SAS 4k SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 


931 GB Mainstream SAS 4k SEF-3 SSD for IBM i AIX, IBM i, and Linux 


1.86 TB Mainstream SAS 4k SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 


ESJV 5B20 | 1.86 TB Mainstream SAS 4k SEF-3 SSD for IBM i AIX, IBM i, and Linux 


ESJW 5B2C | 3.72 TB Mainstream SAS 4k SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 


ESUX 5B2C_ | 3.72 TB Mainstream SAS 4k SEF-3 SSD for IBM i AIX, IBM i, and Linux 


ESJY 5B2E 7.44 TB Mainstream SAS 4k SFF-3 SSD for 18 AIX, IBM i, and Linux 
AIX/Linux 


5B2E | 7.44 TB Mainstream SAS 4k SEF-3 SSD for IBM i AIX, IBM i, and Linux 
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ESJR 


ESJS 5B2B 


ESJT 5B2B 


ESJU 5B20 


esate [an [oecien [ne 
PCle3 x8 3.2 TB NVMe Flash Adapter for IBM i 
PCle3 x8 6.4 TB NVMe Flash Adapter for IBM i 
PCle4 1.6 TB NVMe Flash Adapter x8 for AIX/Linux 
PCle4 3.2 TB NVMe Flash Adapter x8 for AIX/Linux 
PCle4 6.4 TB NVMe Flash Adapter x8 for AIX/Linux 
PCle4 1.6 TB NVMe Flash Adapter x8 for IBM i 


ES62 5B1D | 3.86-4.0 TB 7200 RPM 4K SAS LFF-1 Nearline Disk | 336 AIX, IBM i, and Linux 
Drive (AIX/Linux) 
ES64 5B1F | 7.72-8.0 TB 7200 RPM 4K SAS LFF-1 Nearline Disk | 336 AIX, IBM i, and Linux 
Drive (AIlX/Linux) 
ESD5 59D0 | 600 GB 10K RPM SAS SFF-8 Disk Drive AIX, IBM i, and Linux 
(AIX/Linux) 
ESDB 59E0 300 GB 15K RPM SAS SFF-3 Disk Drive AIX, IBM i, and Linux 
(AIX/Linux) 
ESEU 59D2 571 GB 10K RPM SAS SFF-2 Disk Drive 4K Block - AIX, IBM i, and Linux 
4224 
AIX, IBM i, and Linux 
AIX, IBM i, and Linux 
AIX, IBM i, and Linux 


ESF3 59DA 1.2 TB 10K RPM SAS SFF-2 Disk Drive 4K Block - 
4096 


ESF4 59D3_| 571 GB 10K RPM SAS SFF-3 Disk Drive 4K Block - 
4224 

ESF5 59D3_| 600 GB 10K RPM SAS SFF-3 Disk Drive 4K Block - 
4096 

ESF8 59DB | 1.1 TB 10K RPM SAS SFF-3 Disk Drive 4K Block - 
4224 

ESF9 59DB | 1.2 TB 10K RPM SAS SFF-3 Disk Drive 4K Block - 
4096 
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| 
| 
| 
ESEV 59D2 | 600 GB 10K RPM SAS SFF-2 Disk Drive 4K Block - | 672 
4096 
ESF2 59DA | 1.1 TB 10K RPM SAS SFF-2 Disk Drive 4K Block - | 672 
4224 
| 
. , 
| 
i 


ESFS 59DD | 1.7 TB 10K RPM SAS SFF-2 Disk Drive 4K Block - | 672 AIX, IBM i, and Linux 
4224 

ESFT 59DD | 1.8 TB 10K RPM SAS SFF-2 Disk Drive 4K Block - | 672 AIX, IBM i, and Linux 
4096 

ESFU 59DE | 1.7 TB 10K RPM SAS SFF-3 Disk Drive 4K Block - | 18 AIX, IBM i, and Linux 
4224 

ESFV 59DE | 1.8TB 10K RPM SAS SFF-3 Disk Drive 4K Block - | 18 AIX, IBM i, and Linux 
4096 

ESNJ 5B41 283 GB 15K RPM SAS SFF-3 4k Block Cached Disk | 18 AIX, IBM i, and Linux 
Drive (IBM i) 

ESNK 5B41 300 GB 15K RPM SAS SFF-3 4k Block Cached Disk | 18 AIX, IBM i, and Linux 
Drive (AIX/Linux) 

ESNL 5B43 | 283GB 15K RPM SAS SFF-2 4k Block Cached Disk | 672 AIX, IBM i, and Linux 
Drive (IBM i) 

ESNM 5B43 | 300 GB 15K RPM SAS SFF-2 4k Block Cached Disk | 672 AIX, IBM i, and Linux 
Drive (AlX/Linux) 

ESNN 5B45_ | 571 GB15K RPM SAS SFF-3 4k Block Cached Disk | 18 AIX, IBM i, and Linux 
Drive (IBM i) 

ESNP 5B45_ | 600 GB 15K RPM SAS SFF-3 4k Block Cached Disk | 18 AIX, IBM i, and Linux 
Drive (AIX/Linux) 

ESNQ 5B47 | 571GB15K RPM SAS SFF-2 4k Block Cached Disk | 672 AIX, IBM i, and Linux 
Drive (IBM i) 

ESNR 5B47 | 600 GB 15K RPM SAS SFF-2 4k Block Cached Disk | 672 AIX, IBM i, and Linux 
Drive (AIX/Linux) 


fem [ae [omy corsa cesse ep 
fen [ase [anny wooresnencssasa [ef 
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[Feeds [EON [ecntpion ie‘ 
feson [sets [owmyiwoesowieressm fe | 
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ES 


EQ62 5B1D | Quantity 150 of #ES62 3.86-4.0 TB 7200 rpm 4k 2 
LFF-1 Disk 

5B1F | Quantity 150 of #ES64 7.72-8.0 TB 7200 rpm 4k 2 
LFF-1 Disk 


[ese [260 [oven eooresmnencsrasrra fe | 
fess [sei7 [ouomyoaresno eri cemera [ef 
fesen [2017 [ouwmyrooesmemcsrasra fe | 


The RDX docking station #EUA4 accommodates RDX removable disk cartridges of any 
capacity. The disk is in a protective rugged cartridge enclosure that plugs into the docking 
station. The docking station holds one removable rugged disk drive or cartridge at a time. The 
rugged removable disk cartridge and docking station perform saves, restores, and backups 
similar to a tape drive. This docking station can be an excellent entry capacity or performance 
option. 


EQ64 


EQEU 
EQEV 
EQEY 
EQF2 
EQF3 


The Standalone USB DVD drive (#EUAS) is an optional, stand-alone external USB-DVD 
device. It requires high current at 5 V and must use the front USB 3.0 port. 
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1.6 I/O drawers for Power H922, and Power H924 servers 


If more PCle slots beyond the system node are required, a PCle Gen3 I/O drawer (#EMX0) 
can be attached to the Power H922, and Power H924 servers. 


The EXP12SX SAS Storage Enclosure and EXP24SX SAS Storage Enclosure (#ESLS/ELLS 
or #ESLL/ELLL) provide extra storage capacity. 


1.6.1 PCle Gen3 I/O expansion drawer 


26 


This 19-inch, 4U (4 EIA) enclosure provides PCle Gen@ slots outside of the system unit. It has 
two module bays. One 6-slot fan-out Module (#EMXH or #EMXG) can be placed in each 
module bay. Two 6-slot modules provide a total of 12 PCle Gen3 slots. Each fan-out module is 
connected to a PCle3 Optical Cable adapter that is in the system unit over an active optical 
CXP cable (AOC) pair or CXP copper cable pair. 


The PCle Gen3 I/O Expansion Drawer has two redundant, hot-plug power supplies. Each 
power supply has its own separately ordered power cord. The two power cords plug into a 
power supply conduit that connects to the power supply. The single-phase AC power supply is 
rated at 1030 W and can use 100 - 120 V or 200 - 240 V. If 100 - 120 V is used, the maximum 
is 950 W. It is a best practice that the power supply connects to a power distribution unit 
(PDU) in the rack. Power Systems PDUs are designed for a 200 - 240 V electrical source. 


A blind swap cassette (BSC) is used to house the full-height adapters that go into these slots. 
The BSC is the same BSC that is used with the previous generation server’s 12X attached I/O 
drawers (#5802, #5803, #5877, and #5873). The drawer includes a full set of BSCs, even if 
the BSCs are empty. 


Concurrent repair, and adding or removing PCle adapters is done by HMC-guided menus or 
by operating system support utilities. 


Figure 1-7 shows a PCle Gen I/O expansion drawer. 


Figure 1-7 PCle Gen3 I/O expansion drawer 
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1.6.2 .1/O drawers and usable PCI slots 


Figure 1-8 shows the rear view of the PCle Gen3 I/O expansion drawer that is equipped with 
two PCle3 6-slot fan-out modules with the location codes for the PCle adapter slots. 


P1-C1 P1-C6 P2-C3 P2-C4 


Figure 1-8 Rear view of a PCle Gen3 I/O expansion drawer with PCle slots location codes 


Table 1-11 lists the PCI slots in the PCle Gen3 I/O expansion drawer that is equipped with two 
PCle3 6-slot fan-out modules. 


Table 1-11 PCle slot locations for the PCle Gen3 I/O expansion drawer with two fan-out modules 


[ser towatonense [Destin 
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Consider the following points about the information that is included in Table 1-11 on page 27: 


> All slots support full-length, full-height adapters or short (LP) adapters with a full-height tail 
stock in single-wide, Gen3, BSC. 


> Slots C1 and C4 in each PCle3 6-slot fan-out module are x16 PCle3 buses, and slots C2, 
C3®, C5, and C6 are x8 PCle buses. 


> All slots support enhanced error handling (EEH). 

> All PCle slots are hot-swappable and support concurrent maintenance. 

Table 1-12 lists the maximum number of I/O drawers that are supported and the total number 
of PCI slots that are available. 

Table 1-12 Maximum number of I/O drawers that is supported and total number of PCI slots 


System Maximum number of | Maximum number of | Maximum PCle slots 
I/O expansion I/O fan-out modules 
drawers 


Power H922(1-socket) | 1 1 11 
server 


Power H922(2-socket) | 1 2 19 
server 
Power H924(1-socket) | 1 1 13 
server 
Power H924 (2-socket) | 2 3 26 
server 


1.6.3 EXP24SX SAS Storage Enclosure (#ESLS) and EXP12SX SAS Storage 
Enclosure (#ESLL) 


If you need more disks than are available with the internal disk bays, you can attach more 
external disk subsystems, such as an EXP12SX SAS Storage Enclosure (#ESLL) or 
EXP24SX SAS Storage Enclosure (#ESLS). 


The EXP24SX SAS Storage Enclosure is a storage expansion enclosure with 12 3.5-inch 
large form factor (LFF) SAS bays. It supports up to 12 hot-plug HDDs in only 2 EIA of space in 
a 19-inch rack. The EXP24SX SAS Storage Enclosure SFF bays use LFF Gent (LFF-1) 
trays. The 4 KB sector drives (#4096 or #4224) are supported. SSDs are not supported. 


The EXP24SX SAS Storage Enclosure is a storage expansion enclosure with 24 2.5-inch 
SFF SAS bays. It supports up to 24 hot-plug HDDs or SSDs in only 2 EIA of space ina 
19-inch rack. The EXP24SX SAS Storage Enclosure SFF bays use SFF Gen2 (SFF-2) 
carriers or trays. 


With AIX, Linux, and VIOS, the EXP24SX SAS Storage Enclosure or EXP24SX SAS Storage 
Enclosure can be ordered with four sets of six bays (mode 4), two sets of 12 bays (mode 2), 
or one set of 24-four bays (mode 1). With IBM i, only one set of 24 bays (mode 1) is 
supported. It is possible to change the mode setting in the field by using commands along 
with a documented procedure. 


Important: When changing modes, a skilled, technically qualified person should follow the 
special documented procedures. Improperly changing modes might delete RAID sets, 
which prevents access to data, or allow other partitions to access another partition’s data. 
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Four mini-SAS HD ports on the EXP24SX SAS Storage Enclosure or EXP24SX SAS Storage 
Enclosure are attached to PCle Gen3 SAS adapters or attached to an integrated SAS 
controller in the Power H922, or Power H924 servers. 


The attachment between the EXP24SX SAS Storage Enclosure or EXP24SX SAS Storage 
Enclosure or and the PCle3 SAS adapters or integrated SAS controllers is through SAS 
YO12 or X12 cables. All ends of the YO12 and X12 cables feature mini-SAS HD narrow 
connectors. 


The EXP24SX SAS Storage Enclosure or EXP24SX SAS Storage Enclosure includes 
redundant AC power supplies and two power cords. 


Figure 1-9 shows the EXP24SX Expansion Drawer. 


Figure 1-9 The EXP24SX Expansion Drawer 


Figure 1-10 shows the EXP12SX Expansion Drawer. 


Figure 1-10 The EXP12SX Expansion drawer 
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1.7 System racks 


The Power H922 and Power H924 servers are mounted in the 36U 7014-T00 (#0551), the 
42U 7014-T42 (#0553), or the IBM 42U Slim Rack (7965-94Y) racks. These racks are built to 
the 19-inch EIA 310D standard. 


Ordering information: The racking approach for the initial order must be a 7014-T00, 
7014-T42, or 7965-94Y. If an extra rack is required for I/O expansion drawers as an MES to 
a system, an #0551, #0553, or #ERO5 rack must be ordered. 


You must leave 2U of space at the bottom or top of the rack, depending on the client’s 
cabling preferences, to allow for cabling to exit the rack. 


If a system is installed in a rack or cabinet that is not from IBM, ensure that the rack meets the 
requirements that are described in 1.7.10, “Original equipment manufacturer racks” on 
page 39. 


Responsibility: The client is responsible for ensuring that the installation of the drawer in 
the preferred rack or cabinet results in a configuration that is stable, serviceable, safe, and 
compatible with the drawer requirements for power, cooling, cable management, weight, 
and rail security. 


1.7.1 IBM 7014 Model T00 rack 


The 1.8-meter (71-inch) Model TOO is compatible with past and present Power Systems 
servers. The TOO rack includes the following features: 


> Includes: 


— 36U of usable space 

— Optional removable side panels 

Optional side-to-side mounting hardware for joining multiple racks 
— Increased power distribution and weight capacity 


> Supports AC and DC configurations. 


> Up to four PDUs can be mounted in the PDU bays (see Figure 1-12 on page 35), but 
others can fit inside the rack. For more information, see 1.7.7, “The AC power distribution 
unit and rack content’ on page 34. 


> For the TOO rack, three door options are available: 
— Front Door for 1.8 m Rack (#6068). 


This feature provides an attractive black full-height rack door. The door is steel, with a 
perforated flat front surface. The perforation pattern extends from the bottom to the top 
of the door to enhance ventilation and provide some visibility into the rack. 


— A1.8m Rack Acoustic Door (#6248). 


This feature provides a front and rear rack door that is designed to reduce acoustic 
sound levels in a general business environment. 


— A1.8m Rack Trim Kit (#6263). 
If no front door is used in the rack, this feature provides a decorative trim kit for the front. 
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>» Ruggedized Rack Feature 


For enhanced rigidity and stability of the rack, the optional Ruggedized Rack Feature 
(#6080) provides more hardware that reinforces the rack and anchors it to the floor. This 
hardware is designed primarily for use in locations where earthquakes are a concern. The 
feature includes a large steel brace or truss that bolts into the rear of the rack. 


It is hinged on the left side so it can swing out of the way for easy access to the rack 
drawers when necessary. The Ruggedized Rack Feature also includes hardware for 
bolting the rack to a concrete floor or a similar surface, and bolt-in steel filler panels for any 
unoccupied spaces in the rack. 


> Weights: 
— TOO base empty rack: 244 kg (535 Ib.). 
— TOO full rack: 816 kg (1795 Ib.). 
— Maximum weight of drawers is 572 kg (1260 Ib.). 


— Maximum weight of drawers in a zone 4 earthquake environment is 490 kg (1080 Ib.). 
This number equates to 13.6 kg (80 Ib.) per EIA. 


Important: If more weight is added to the top of the rack, for example, adding feature code 
#6117, the 490 kg (1080 Ib.) must be reduced by the weight of the addition. In our example, 
feature code #6117 weighs approximately 45 kg (100 Ib.), so the new maximum weight of 
drawers that the rack can support in a zone 4 earthquake environment is 445 kg (980 Ib.). 
In the zone 4 earthquake environment, the rack must be configured starting with the 
heavier drawers at the bottom of the rack. 


1.7.2 IBM 7014 Model T42 rack 


The 2.0-meter (79.3-inch) Model T42 addresses the client requirement for a tall enclosure to 
house the maximum amount of equipment in the smallest possible floor space. The following 
features are for the model T42 rack (which differ from the model TOO): 


> The T42 rack has 42U of usable space (6U of extra space). 
> The model T42 supports AC power only. 
> Weights: 


— T42 base empty rack: 261 kg (575 Ib.) 
— T42 full rack: 930 kg (2045 Ib.) 
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The available door options for the Model T42 rack are shown in Figure 1-11. 


Trim kit Acoustic doors 780 logo Optional 
(no front door) Plain front door (front and rear) front door front door 
FC 6272 FC 6069 FC 6249 FC 6250 FC ERG7 


Figure 1-11 Door options for the T42 rack 


Consider the following points about the available door options: 
> The 2.0 m Rack Trim Kit (#6272) is used if no front door is used in the rack. 


> The Front Door for a 2.0 m Rack (#6069) is made of steel, with a perforated flat front 
surface. The perforation pattern extends from the bottom to the top of the door to enhance 
ventilation and provide some visibility into the rack. This door is non-acoustic and is 
approximately 25 mm (1 in.) deep. 


> The 2.0 m Rack Acoustic Door (#6249) consists of a front and rear door to reduce noise by 
approximately 6 dB(A). It is approximately 191 mm (7.5 in.) deep. 


> Feature code #ERG7 provides an attractive black full-height rack door. The door is steel, 
with a perforated flat front surface. The perforation pattern extends from the bottom to the 
top of the door to enhance ventilation and provide some visibility into the rack. The 
non-acoustic door is approximately 134 mm (5.3 in.) deep. 


Rear Door Heat Exchanger 


To lead away more heat, a special door that is named the Rear Door Heat Exchanger (RDHX) 
(#6858) is available. This door replaces the standard rear door on the rack. Copper tubes that 
are attached to the rear door circulate chilled water, which is provided by the customer. The 
chilled water removes heat from the exhaust air that is blown through the servers and 
attachments that are mounted in the rack. With industry-standard quick couplings, the water 
lines in the door attach to the customer-supplied secondary water loop. 


For more information about planning for the installation of the IBM Rear Door Heat 
Exchanger, see IBM Knowledge Center. 
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1.7.3 IBM 42U Slim Rack 7965-94Y 


The 2.0-meter (79-inch) Model 7965-94Y is compatible with past and present Power Systems 
servers and provides an excellent 19-inch rack enclosure for your data center. Its 600 mm 
(23.6 in.) width combined with its 1100 mm (43.3 in.) depth plus its 42 EIA enclosure capacity 
provides great footprint efficiency for your systems and allows it to be easily placed on 
standard 24-inch floor tiles. 


The IBM 42U Slim Rack has a lockable perforated front steel door that provides ventilation, 
physical security, and visibility of indicator lights in the installed equipment within. In the rear, 
a lockable perforated rear steel door (#ECO2) or a lockable RDHX (1164-95X) is used. 


Lockable optional side panels (#EC03) increase the rack’s aesthetics, help control airflow 
through the rack, and provide physical security. Multiple 42U Slim Racks can be bolted 
together to create a rack suite (#ECO04). 


Up to six optional 1U PDUs can be placed vertically in the sides of the rack. More PDUs can 
be placed horizontally, but they each use 1U of space in this position. 


1.7.4 1.8 Meter Rack (#0551) 


The 1.8 Meter Rack (#0551) is a 36 EIA unit rack. The rack that is delivered as #0551 is the 
same rack that is delivered when you order the 7014-T00 rack. The included features might 
vary. Certain features that are delivered as part of the 7014-T00 must be ordered separately 
with #0551. 


1.7.5 2.0 Meter Rack (#0553) 


The 2.0 Meter Rack (#0553) is a 42 EIA unit rack. The rack that is delivered as #0553 is the 
same rack that is delivered when you order the 7014-T42 rack. The included features might 
vary. Certain features that are delivered as part of the 7014-T42 must be ordered separately 
with #0553. 


1.7.6 Rack (#ER05) 


This feature provides a 19-inch, 2.0-meter high rack with 42 EIA units of total space for 
installing a rack-mounted central electronics complex (CEC) or expansion units. The 600 mm 
wide rack fits within a data center’s 24-inch floor tiles and provides better thermal and cable 
management capabilities. The following features are required on #ERO5: 


> #ECO01 front door 
> #ECO2 rear door or #ECO5 RDHxX indicator 


PDUs on the rack are optional. Each #7196 and #7189 PDU uses one of six vertical mounting 
bays. Each PDU beyond four uses 1U of rack space. 


If you order Power Systems equipment in an MES order, use the equivalent rack #ERO5 
instead of 7965-94Y so IBM Manufacturing can include the hardware in the rack. 
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1.7.7 The AC power distribution unit and rack content 


For rack models TOO and T42, 12-outlet PDUs are available, which include the AC PDUs 
#9188 and #7188 and the AC Intelligent PDU+ (iPDU) #5889 and #7109. The iPDUs+ (#5889 
and #7109) are identical to #9188 and #7188 PDUs, but are equipped with one Ethernet port, 
one console serial port, and one RS232 serial port for power monitoring. 


The PDUs have 12 client-usable IEC 320-C13 outlets. Six groups of two outlets are fed by six 
circuit breakers. Each outlet is rated up to 10 A, but each group of two outlets is fed from one 
15 A circuit breaker. 


High-function PDUs provide more electrical power per PDU and offer better “PDU footprint” 
efficiency. In addition, they are iPDUs that provide insight to power usage by receptacle and 
also provide remote power on/off capability for easier support by individual receptacle. The 
PDUs can be ordered as #EPTJ, #EPTL, #EPTN, #EPTQ, #ECJJ, #ECJUL, #ECUN, and 
#ECJQ. 


High-function PDU feature codes are listed in Table 1-13. 


Table 1-13 Available high-function PDUs 
1-phase or 3-phase depending 3-phase 208 V depending on 
on country wiring standards country wiring standards 
Old New Old New 
generation generation generation generation 


Nine C19 receptacles #EPTJ #ECJJ #EPTL #ECJL 
Twelve C13 receptacles | #EPTN #ECJN #EPTQ #ECJQ 


The following new generation high-function PDUs are available: 
> High Function 9xC19 PDU plus (#ECJJ) 


This intelligent, switched 200 - 240 volt AC PDU plus features nine C19 receptacles on the 
front of the PDU. The PDU is mounted at the rear of the rack, making the nine C19 
receptacles easily accessible. For comparison, this feature code is most similar to the 
earlier generation #EPTJ PDU. 


> High Function 9xC19 PDU plus 3-Phase (#ECJL) 


This intelligent, switched 208 volt 3-phase AC PDU plus features nine C19 receptacles on 
the front of the PDU. The PDU is mounted at the rear of the rack making the nine C19 
receptacles easily accessible. For comparison, this feature code is most similar to the 
earlier generation #EPTL PDU. 


> High Function 12xC13 PDU plus (#ECJN) 
This intelligent, switched 200 - 240 volt AC PDU plus features 12 C13 receptacles on the 
front of the PDU. The PDU is mounted at the rear of the rack making the 12 C13 


receptacles easily accessible. For comparison, this feature code is most similar to the 
earlier generation #EPTN PDU. 


> High Function 12xC13 PDU plus 3-Phase (#ECJQ) 


This intelligent, switched 208 volt 3-phase AC PDU plus features 12 C13 receptacles on 
the front of the PDU. The PDU is mounted at the rear of the rack making the 12 C13 
receptacles easily accessible. For comparison, this feature code is most similar to the 
earlier generation #EPTQ PDU. 
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Four PDUs can be mounted vertically in the back of the TOO and T42 racks. Figure 1-12 
shows the placement of the four vertically mounted PDUs. At the rear of the rack, two more 
PDUs can be installed horizontally in the TOO rack and three in the T42 rack. The four vertical 
mounting locations are filled first in the TOO and T42 racks. Mounting PDUs horizontally uses 
1U per PDU and reduces the space that is available for other racked components. When 
mounting PDUs horizontally, the preferred approach is to use fillers in the EIA units that are 
occupied by these PDUs to facilitate proper air-flow and ventilation in the rack. 


Rack Rear View 
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Figure 1-12 Power distribution unit placement and view 


The PDU receives power through a UTG0247 power-line connector. Each PDU requires one 
PDU-to-wall power cord. Various power cord features are available for various countries and 
applications by varying the PDU-to-wall power cord, which must be ordered separately. 


Each power cord provides the unique design characteristics for the specific power 
requirements. To match new power requirements and save previous investments, these 
power cords can be requested with an initial order of the rack or with a later upgrade of the 
rack features. 


Chapter 1. General description 35 


Table 1-14 lists the available wall power cord options for the PDU and iPDU features, which 
must be ordered separately. 


Table 1-14 Wall power cord options for the PDU and iPDU features 


Feature Rated voltage Rated amperage Geography 

code (V AC) 

#6653 IEC 309, 230 3 16 A/phase Internationally available 
3P+N+G, 16A 

#6489 IEC309 230 3 32 A/phase EMEA 
3P+N+G, 32 A 


#6654 NEMA L6-30 200 - 208, 240 
#6655 RS 3750DP 200-208, 240 | 1 
(watertight) 
#6656 IEC 309, 230 1 
P+N+G, 32 A 


#6492 IEC 309, 2P+G, 200 - 208, 240 1 
60A 

#6491 IEC 309, P+N+G, | 230 1 EMEA 
63 A 


#6667 PDL 56P532 230 - 240 Australia and New Zealand 
#7196 IEC 320 200 - 208, 240 US, Canada, LA, and Japan 


Notes: Ensure that the suitable power cord feature is configured to support the power that 
is being supplied. Based on the power cord that is used, the PDU can supply 

4.8 - 19.2 kVA. The power of all the drawers that are plugged into the PDU must not 
exceed the power cord limitation. 


The Universal PDUs are compatible with previous models. 


To better enable electrical redundancy, each server has two power supplies that must be 
connected to separate PDUs, which are not included in the base order. 


For maximum availability, a preferred approach is to connect power cords from the same 
system to two separate PDUs in the rack, and to connect each PDU to independent power 
sources. 


For more information about power requirements and power cords for the 7014 and 7965-94Y 
racks, see IBM Knowledge Center. 
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1.7.8 Rack-mounting rules 


Consider the following primary rules when you mount the system into a rack: 


> The system is placed at any location in the rack. For rack stability, start filling a rack from 
the bottom. 


> Any remaining space in the rack can be used to install other systems or peripheral devices 
if the maximum permissible weight of the rack is not exceeded and the installation rules for 
these devices are followed. 


> Before placing the system into the service position, be sure to follow the rack 
manufacturer's safety instructions regarding rack stability. 


Order information: The racking approach for the initial order must be a 7014-T00, 
7014-T42, or 7965-94Y. If an extra rack is required for I/O expansion drawers as an MES to 
an existing system, an #0551, #0553, or #ERO5 rack must be ordered. 


You must leave 2U of space at the bottom or top of the rack (depending on the client’s 
cabling preferences) to allow for cabling to exit the rack. 


1.7.9 Useful rack additions 


This section highlights several rack other solutions for Power Systems rack-based systems. 


IBM System Storage 7226 Model 1U3 Multi-Media Enclosure 


The IBM System Storage 7226 Model 1U3 Multi-Media Enclosure can accommodate up to 
two tape drives, two RDX removable disk drive docking stations, or up to four DVD-RAM 
drives. 


The IBM System Storage 7226 Multi-Media Enclosure supports LTO Ultrium, DVD-RAM, and 
RDX removable storage requirements on the following IBM processor-based systems: 


> IBM POWER7® 
> IBM POWER8® 
> IBM POWERQ 


The IBM System Storage 7226 Multi-Media Enclosure offers an expansive number of drive 
feature options, as listed in Table 1-15. 


Table 1-15 Supported drive features for the 7226-1U3 


fem ess ES 
DVD-RAM USB Optical Drive 


#8341 LTO Ultrium 6 Half High SAS Tape Drive Available 
#8441 LTO Ultrium 7 Half High SAS Tape Drive Available 


#8348 LTO Ultrium 6 Half High Fibre Tape Drive Available 
#8546 LTO Ultrium 8 Half High Fibre Tape Drive Available 
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#EU03 RDX 3.0 Removable Disk Docking Station Available 


The options include the following features: 
> LTO Ultrium 6 Half-High 2.5 TB SAS and Tape Drive 


With a data transfer rate up to 320 MBps (assuming a 2.5:1 compression), the LTO Ultrium 
6 drive is read/write compatible with LTO Ultrium 6 and 5 media, and read-only compatible 
with LTO Ultrium 4. By using data compression, an LTO-6 cartridge can store up to 

6.25 TB of data. 


> LTO Ultrium 7 drive 


Offers a data rate of up to 300 MBps with compression. It also provides read/write 
compatibility with Ultrium 7 and Ultrium 6 media formats, and read-only compatibility with 
Ultrium 5 media formats. By using data compression, an LTO-7 cartridge can store up to 
15 TB of data. 


> LTO Ultrium 8 drive 


Offers a data rate of up to 300 MBps with compression. It also provides read/write 
compatibility with Ultrium 8 and Ultrium 7 media formats. It is not read or write compatible 
with other Ultrium media formats. By using data compression, an LTO-8 cartridge can 
store up to 30 TB of data. 


>» DVD-RAM 


The 9.4 GB SAS Slim Optical Drive with an SAS and USB interface option is compatible 
with most standard DVD disks. 


> RDX removable disk drives 


The RDX USB docking station is compatible with most RDX removable disk drive 
cartridges when it is used in the same OS. The 7226 offers the following RDX removable 
drive capacity options: 

— 500 GB (#1107) 


— 1.0 TB (#EU01) 
— 2.0 TB (#EU2T) 


Removable RDX drives are in a rugged cartridge that inserts in to an RDX removable (USB) 
disk docking station (#1103 or #EU03). RDX drives are compatible with docking stations, 
which are installed internally in POWER6, IBM POWER6+, POWER7, IBM POWER7z4, 
POWERS8, and POWERS processor-based servers, where applicable. 


Figure 1-13 shows the IBM System Storage 7226 Multi-Media Enclosure. 


Figure 1-13 IBM System Storage 7226 Multi-Media Enclosure 
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The IBM System Storage 7226 Multi-Media Enclosure offers customer-replaceable unit 
(CRU) maintenance service to help make the installation or replacement of new drives 
efficient. Other 7226 components are also designed for CRU maintenance. 


The IBM System Storage 7226 Multi-Media Enclosure is compatible with most POWER6, 
POWER6+, POWER7, POWER7+, POWER8, and POWER9 processor-based systems that 
offer current level AIX, IBM i, and Linux operating systems. 


For more information about host software versions and release levels that support the IBM 
System Storage 7226 Multi-Media Enclosure, see System Storage Interoperation Center 
(SSIC). 


Note: Any of the 7216-1U3 and 7214-1U2 multimedia drawers also are supported. 


1.7.10 Original equipment manufacturer racks 


The system can be installed in a suitable original equipment manufacturer (OEM) rack if that 
the rack conforms to the EIA-310-D standard for 19-inch racks. This standard is published by 
the Electrical Industries Alliance (EIA). For more information, see IBM Knowledge Center. 


Consider the following points: 


> The front rack opening must be 451 mm wide + 0.75 mm (17.75 in. + 0.03 in.), and the 
rail-mounting holes must be 465 mm + 0.8 mm (18.3 in. + 0.03 in.) apart on-center (that is, 
the horizontal width between the vertical columns of holes on the two front-mounting 
flanges and on the two rear-mounting flanges). Figure 1-14 shows a top view of the 


specification dimensions. 
Top Front of Rack 


Top Front of Rack 


6.75mm min 
15.9mm 


15.9mm 


12.7mm 
15.9mm 


15.9mm 
12.7mm 


ko 450 +/- 0.75mm | 


Rack Front Opening 
465 +/- 0.8mm ——_> 
Rack Mounting Holes Center-to-Center 


Figure 1-14 Top view of the rack specification dimensions (not specific to IBM) 


EIA Hole Spacing 


Hole Diameter = 
7.1 +/- 0.1mm 
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> The vertical distance between the mounting holes must consist of sets of three holes 
spaced (from bottom to top) 15.9 mm (0.625 in.), 15.9 mm (0.625 in.), and 12.67 mm (0.5 
in.) on-center, making each three-hole set of vertical hole spacing 44.45 mm (1.75 in.) 
apart on center. Rail-mounting holes must be 7.1 mm + 0.1 mm (0.28 in. + 0.004 in.) in 
diameter. Figure 1-15 shows the top front specification dimensions. 


Top Front of Rack 


Top Front of Rack 


6.75mm min 
15.9mm 


15.9mm 
12.7mm 
15.9mm 
15.9mm 
12.7mm 


450 +/- 0.75mm ——+| 


‘ - Rack Front Opening 
Hole Diameter 465 +/- 0.8mm 


7.1 +/- 0.4mm ; 
Rack Mounting Holes Center-to-Center 


EIA Hole Spacing 


Figure 1-15 Rack specification dimensions: Top front view 


1.8 Hardware Management Console 


This section describes the HMCs that are available for Power Systems servers. 


1.8.1 New features 


40 


The HMCs include the following new features: 


> New HMCs are now based on a platform with POWER processors. 
> Intel x86-based HMCs are supported, but are no longer available. 
> Virtual HMCs (vHMCs) are available for x86 and Power Systems virtual environments. 
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1.8.2 Hardware Management Console overview 


Administrators can use the HMC, which is a dedicated appliance, to configure and manage 
system resources on Power Systems servers. You can use the HMC to manage the servers 
through a GUI, command-line interface (CLI), or a REST API interface. The HMC provides 
basic virtualization management support for configuring LPARs and dynamic resource 
allocation, including processor and memory settings for selected Power Systems servers. 


The HMC also supports advanced service functions, including guided repair and verification, 
concurrent firmware updates for managed systems, and around-the-clock error reporting 
through IBM Electronic Service Agent (IBM ESA) for faster support. 


The HMC management features help improve a server usage, simplify systems management, 
and accelerate provisioning of server resources by using PowerVM virtualization technology. 


The HMC is available as a hardware appliance or as a virtual machine (VHMC) that is hosted 
on a x86 or the Power platform. The Power H922, and Power H924 servers support several 
service environments, including attachment to one or more HMCs. This configuration is 
recommended for servers that support multiple LPARs with a dedicated resource or virtual 
/0. 


The following architectures are available for hosting the HMC: 


> x86-based HMCs: 7042-CR7, CR8, or CRY 
> POWER based HMC: 7063-CR1 
>» vHMC on x86 or Power Systems LPARs 


Hardware support for CRUs comes standard with the HMC. In addition, users can upgrade 
this support level to IBM onsite support to be consistent with other Power Systems servers. 


Note: An HMC or vHMC is required to manage the Power H 922, and Power H924. 


For more information about vVHMC, see Virtual HMC Appliance (VHMC) Overview. 


Figure 1-16 shows the HMC model selections and tier updates. 


Old HMC models 


- ae 
= 
- fee 


Old HMC virtual appliance 


HMC virtual appliance for x86 systems 
HMC virtual appliance for 
POWER processor-based systems 


New HMC virtual appliance 


Figure 1-16 HMC model selections 
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1.8.3 Hardware Management Console code level 


The HMC code must be running at Version 9 Release 1 modification 940 service pack 1 
(V9R1M941) or later when you manage the Power H922 and Power H924 servers. 


If you are attaching an HMC to a new server or adding a function to an existing server that 
requires a firmware update, the HMC machine code might need to be updated to support the 
server firmware level. In a dual-HMC configuration, both HMCs must be at the same version 
and release of the HMC code. 


To determine the HMC machine code level that is required for the firmware level on any 
server, see the Fix Level Recommendation Tool (FLRT) web page on or after the planned 
availability date for this product. 


FLRT identifies the correct HMC machine code for the selected system firmware level. 


Note: Consider the following points: 


> Access to firmware and machine code updates is conditional on entitlement and license 
validation in accordance with IBM policy and practice. IBM might verify entitlement 
through customer number, serial number electronic restrictions, or any other means or 
methods that are used by IBM at its discretion. 


> HMC V9 supports only the Enhanced+ version of the GUI. The Classic version is no 
longer available. 


> HMC V9R1.911.0 added support for managing IBM OpenPOWER systems. The same 
HMC that is used to manage Flexible Service Processor (FSP)-based enterprise 
systems can manage the baseboard management controller (BMC) based Power 
Systems AC and Power Systems LC servers. This support provides a consistent and 
consolidated hardware management solution. 


> HMC V9 supports connections to servers that are based on IBM servers that are based 
on POWER9Y, POWER8, and POWER7 processors. POWER6G servers or earlier are not 
supported. 


1.8.4 Two architectures of Hardware Management Console 


Two options are available for the HMC hardware: The earlier Intel-based HMCs, and the 
newer model that is based on an IBM POWER8® processor. The x86-based HMCs are 
withdrawn from marketing but still supported as an option for managing the Power H922 and 
Power H924 servers. 


A mix of Intel and POWER processor-based HMCs can be used for managing POWER9 
servers if the firmware level is the same. 


As a best practice, use 7063-CR1 or a vHMC for the server management. 


Intel-based HMCs 
The following Intel-based HMCs still support V9R1 firmware: 


> 7042-CR9 
> 7042-CR8 
> 7042-CR7 
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POWERS processor-based HMC 
The POWER8 processor-based HMC is machine type and model 7063-CR1. It features the 
following specifications: 


vvvvvvvvvy 


1.8.5 Virtual 


1U base configuration 

IBM POWERS8 120 W 6-core CPU 

32 GB (4 x 8 GB) of DDR4 system memory 

Two 2-TB SATA LFF 3.5-inch HDD RAID 1 disks 

Rail bracket option for round hole rack mounts 

Two USB 3.0 hub ports in the front of the server 

Two USB 3.0 hub ports in the rear of the server 
Redundant 1 kW power supplies 

Four 10-Gb Ethernet Ports (RJ-45) (10 Gb/1 Gb/100 Mb) 
One 1-Gb Ethernet port for management (BMC) 


Note: System administrators can remotely start or stop a 7063-CR1 HMC by using 
ipmitool or the WebUI. 


Hardware Management Console 


The virtual appliance can be installed in x86 or POWER virtualized infrastructures. 


VHMC for x86 hypervisors 
The supported hypervisors for x86 are: 


> 
> 
> 


VMware ESXi 6.0 or higher 
Kernel-based virtual machine (KVM) on Red Hat Enterprise Linux 7 or higher 
Xen 4.2 or higher on SUSE Linux Enterprise Server 12 


The following minimum requirements must be met to run the HMC virtual appliance: 


vvvy 


16 GB of memory 

Four virtual processors 

Two network interfaces (a maximum of four are allowed) 
One disk drive that contains 500 GB of available disk space 


VHMC for POWER hypervisor 
PowerVM is the only supported hypervisor for POWER machines. 


The following minimum requirements must be to run the HMC virtual appliance: 


> 
> 
> 


16 GB of memory is recommended 
One processing unit and four shared virtual processors in capped sharing mode 
160 GB of disk space (500 GB is recommended) 


Note: You cannot manage the server that hosts the HMC virtual appliance. 
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1.8.6 Connectivity to POWER9 processor-based systems 


POWERS processor-based servers and their predecessor systems that are managed by an 
HMC require Ethernet connectivity between the HMC and the server's service processor. 
Also, to perform an operation on an LPAR,, start Live Partition Mobility (LPM), or perform 
PowerVM Active Memory Sharing operations, you must have an Ethernet link to the managed 
partitions. A minimum of two Ethernet ports are needed on the HMC to provide such 
connectivity. 


For the HMC to communicate with the managed server, ethO of the HMC must be connected 
to the HMC1 or HMC2 ports of the managed server (although other network configurations 
are possible). You can attach a second HMC to the remaining HMC port of the server for 
redundancy. The two HMC ports must be addressed by two separate subnets. 


Figure 1-17 shows a simple network configuration to enable the connection from the HMC to 
the server and to allow for dynamic LPAR operations. For more information about the HMC 
and the possible network connections, see /BM Power Systems HMC Implementation and 
Usage Guide, SG24-7491. 


Management LAN 


LPAR 
2 


LPAR y LPAR | LPAR 
1 a n 


Power Systems server 


Figure 1-17 Network connections from the HMC to service processor and LPARs 


By default, the service processor HMC ports are configured for dynamic IP address 
allocation. The HMC can be configured as a DHCP server, which provides an IP address at 
the time that the managed server is powered on. In this case, the FSP is allocated an IP 
address from a set of address ranges that is predefined in the HMC software. 


If the service processor of the managed server does not receive a DHCP reply before 
timeout, predefined IP addresses are set up on both ports. Static IP address allocation is also 
an option and can be configured by using the Advanced System Management Interface 
(ASMI) menus. 
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Note: The two service processor HMC ports have the following features: 
> 1 Gbps connection speed. 


> Visible only to the service processor. They can be used to attach the server to an HMC 
or to access the ASMI options from a client directly from a client web browser. 


> Use the following network configuration if no IP addresses are set: 


— Service processor ethO (HMC1 port): 169.254.2.147 with netmask 255.255.255.0 
— Service processor eth1 (HMC2 port): 169.254.3.147 with netmask 255.255.255.0 


1.8.7 High availability Hardware Management Console configuration 


The HMC is an important hardware component. Although Power Systems servers and their 
hosted partitions can continue to operate when the managing HMC becomes unavailable, 
specific operations, such as dynamic LPAR, partition migration that uses PowerVM LPM, or 
the creation of a partition, cannot be performed without the HMC. Power Systems servers 
might have two HMCs attached to a system, which provides redundancy if one of the HMCs is 
unavailable. 


To achieve HMC redundancy for a POWER9 processor-based server, the server must be 
connected to two HMCs. Consider the following points: 

>» The HMCs must be running the same level of HMC code. 

> The HMCs must use different subnets to connect to the service processor. 


> The HMCs must communicate with the server's partitions over a public network to allow for 
full synchronization and functions. 
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Figure 1-18 shows one possible HA HMC configuration that manages two servers. 


LAN 1 - Hardware management 
network for first FSP ports (private) 


rLE 


LAN 2 - Hardware management 
network for second FSP ports (private), 
different network from LAN 1 


LAN 3 — Open network for HMC access 
and DLPAR operations 


LPAR B1 


coe 


Figure 1-18 Highly available HMC networking example 


Each HMC is connected to one FSP port of each managed server. 


For simplicity, only the hardware management networks (LAN1 and LAN2) are HA. However, 
the open network (LAN3) can be made HA by using a similar concept and adding a second 
network between the partitions and HMCs. 


For more information about redundant HMCs, see /BM Power Systems HMC Implementation 
and Usage Guide, SG24-7491. 
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Architecture and technical 
overview 


This chapter describes the overall system architecture for the IBM Power System H922 
(9223-22H) and the IBM Power System H924 (9223-42H) servers. The bandwidths that are 
provided throughout the chapter are theoretical maximums that are used for reference. 


The speeds that are shown are at an individual component level. Multiple components and 
application implementations are key to achieving the preferred performance. 


Always complete the performance sizing at the application workload environment level and 
evaluate performance by using real-world performance measurements and production 
workloads. 


The figures that are used in this chapter provide an overview of the architectural 
enhancements of the new systems, compared with previous release of Power H922 and 
Power H924 servers: 


> Gen4 PCle switches, which enable all I/O slots that are PCle Gen4 capable and double 
the number of PCle data lanes to the NVMe backplanes 


> Use U.2 NVMe Flash Adapter as internal storage 
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Figure 2-1 shows the IBM Power H924 logical system diagram. 
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Figure 2-1 IBM Power H924 logical system diagram 
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Figure 2-2 shows the IBM Power H922 logical system diagram. 
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Figure 2-2 IBM Power H922 logical system diagram 


Note: For slots C6 and C12, check adapter placement rules when x16 cards are used in 


these slots to obtain the best throughput. 
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This chapter includes the following topics: 


2.1, “IBM POWER$ processor’ on page 51 
2.2, “Memory subsystem” on page 61 

2.3, “System buses” on page 70 

2.4, “Internal I/O subsystem” on page 73 

2.5, “PCle adapters” on page 76 

2.6, “Internal storage” on page 84 

2.7, “External 1O subsystems” on page 100 
2.8, “External disk subsystems” on page 103 
2.9, “Operating system support” on page 107 
2.10, “Power Systems reliability, availability, and serviceability capabilities” on page 109 
2.11, “Manageability” on page 111 
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2.1 IBM POWERS processor 


This section introduces the POWER9 processor as it is used in the Power H924 and Power 
H922 servers and describes its main characteristics and features in general. 


2.1.1 POWER9 processor overview 


This section describes the architectural design of the POWER39 processor. 


The servers are offered with various numbers of cores that are activated, and a selection of 
clock frequencies, so that IBM can make offerings at several price points. Customers can 
select a specific server (or servers) to fit their budget and performance requirements. 


POWERS is a modular execution slice microarchitecture in which each execution slice 
features its own instruction queue, register files, execution engine, and cache port. It is 
implemented as a 64-bit, 8-byte doubleword data flow. 


The POWER39 architecture supports two core variants: An SMT4 core that supports up to four 
threads with four execution slices and an SMT8 core that supports up to eight threads with 
eight execution slices. 


The SMT8 core is optimized for larger partitions and especially suited for transactional 
workloads and data-centric business analytics applications that exhibit multiple threads with 
strong thread interaction. 


The Power H922 and Power H924 servers all use the SMT8 core variant with 4, 8, 10, 11, or 
12 active cores per installed processor module, depending on the specific processor feature 
codes that are selected. 


Note: The 12 active cores option is available for Power H924 model only. The four active 
cores option is available for Power H922 model only in a one-socket configuration. 


The POWERS processor is a Single Chip Module (SCM) that is manufactured on the IBM 
14-nm FinFET Silicon-On-Insulator (SOI) architecture. Each module is 68.5 mm x 68.5 mm 
and contains 8 billion transistors 
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As shown in Figure 2-3, the chip contains 12 cores, two memory controllers, Peripheral 
Component Interconnect Express (PCle) Gen 4 I/O controllers, and an interconnection 
system that connects all components within the chip at 7 TBps. Each core has 512 KB of L2 
cache and 10 MB of L3 embedded dynamic random access memory (eDRAM). The 
interconnect also extends through module and system board technology to other POWER9 
processors in addition to DDR4 memory and various I/O devices. 
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Figure 2-3. The 12-core POWER9 processor chip 


The POWERS processor has eight memory channels. Each channel supports up to two 
DDR4 DIMM slots. The Power H924 and the Power H922 servers can support up to 4 TB of 
memory in two-socket configuration. 


2.1.2 POWER9 processor features 


The POWER9 processor includes an embedded logic that provides the following features: 
> Two memory controllers that support direct-attached DDR4 memory. 

> Three integrated PCle Gen 4 controllers with a total of 42 lanes running at 16 Gbps. 

> Two 842 compression/decompression engines for 4 K and 64 K memory pages. 


> Native Gzip compression and decompression accelerator. 
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> Two cryptographic accelerator engines that support Advanced Encryption Standard (AES) 
and Secure Hash Algorithm (SHA) operations. 


> Random number generator (RNG). 
Table 2-1 lists the POWER9 processor features. 


Table 2-1 Summary of the POWER9 processor technology 


Technology POWERS processor 


Fabrication technology 14-nm lithography 
Copper interconnect 
SOl 
eDRAM 


2.1.3 POWERS processor core 


The POWER39 processor core is a 64-bit implementation of the IBM Power Instruction Set 
Architecture (ISA) Version 3.0. It includes the following features: 


> Multi-threaded design, which is capable of up to eight-way simultaneous multithreading 
(SMT8). 


> 64-KB, 8-way set-associative L1 instruction cache. 
> 64-KB, 8-way set-associative L1 data cache. 


> Enhanced prefetch, with instruction speculation awareness and data prefetch depth 
awareness. 


> Enhanced branch prediction that uses local and global prediction tables with a selector 
table to choose the best predictor. 


> Improved out-of-order execution. 
> Two symmetric fixed-point execution units. 


> Two symmetric load/store units and two load units, all four of which can also run simple 
fixed-point instructions. 


> An integrated, multi-pipeline vector-scalar floating point unit for running scalar and 
SIMD-type instructions, including the Vector Multimedia eXtension (VMX) instruction set 
and the improved Vector Scalar eXtension (VSX) instruction set, which is capable of up to 
16 floating point operations per cycle (eight double precision or 16 single precision). 


> In-core AES encryption capability. 


> Hardware data prefetching with 16 independent data streams and software control. 


Chapter 2. Architecture and technical overview 53 


> Hardware decimal floating point (DFP) capability. 


For more information about Power ISA Version 3.0, see this web page. 


2.1.4 Simultaneous multithreading 


POWERS processor advancements in multi-core and multithread scaling are remarkable. A 
significant performance opportunity comes from parallelizing workloads to enable the full 
potential of the microprocessor, and the large memory bandwidth. Application scaling is 
influenced by multi-core and multithread technology. 


SMT enables a single physical processor core to simultaneously dispatch instructions from 
more than one hardware thread context. With SMT, each POWERS core can present eight 
hardware threads. Because multiple hardware threads are used per physical processor core, 
more instructions can run at the same time. 


SMT is primarily beneficial in commercial environments where the speed of an individual 
transaction is not as critical as the total number of transactions that are performed. SMT 
typically increases the throughput of workloads with large or frequently changing working 
sets, such as database servers and web servers. 


The Power H922, and Power H924 servers support the full range of SMT modes that are 
available with POWER39 processors: 


Single thread mode (SMTO) 
Dual thread mode (SMT2) 
Four-way SMT mode (SMT4) 
Eight-way SMT mode (SMT8) 


vvvy 


2.1.5 Processor feature codes 
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This section provides more information about the available processor card feature codes for 
Power H922 and Power H924 servers. For each of the named servers, the number of 
functional cores, their typical frequency range, and the processor core activation feature 
codes that are related to the supported processor card features are listed. 


Power H922 processor feature codes 

The Power H922 (9223-22S) server is a 2-socket server with up to 22 active cores. Four 
different POWER9 processor card features with 4, 8, 10, or 11 functional cores are available. 
You cannot use different processor card features within a Power H922 server. The smallest 
POWERS 4-core processor card allows only for one-socket configurations. All other 
processor card features can be used in one-socket or two-socket configurations. It is possible 
to add a second socket as a miscellaneous equipment specification (MES) upgrade order toa 
system that was initially ordered with only one 8-core, 10-core, or 11-core processor card. 


All functional cores on a processor card must be activated through related processor core 
activation features. 
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Table 2-2 lists the number of functional cores, their typical frequency range, and the 
processor core activation feature codes for each of the available processor card feature codes 
of a Power H922 server. 


Table 2-2 Processor feature code specification for the Power H922 server 


Processorcard | Configuration Number of Typical Processor core 
feature code options cores frequency range | activation 
[GHz] feature code 


#EP76 1-socket only 28-38 EP7C 


Power H924 processor feature codes 

The Power H924 (9223-42S) server is a powerful 2-socket server with up to 24 cores. Four 
different POWER processor card features with 8, 10, 11, or 12 functional cores are available. 
You cannot use different processor card features within a Power H924 server. All processor 
card features can be used in one-socket or two-socket configurations. It is possible to add a 
second socket as a MES upgrade order to a system that was initially ordered with only one 
8-core, 10-core, 11-core, or 12-core processor card. 


All functional cores on a processor card must be activated through related processor core 
activation features. 


Table 2-3 lists the number of functional cores, their typical frequency range, and the 
processor core activation feature codes for each of the available processor card feature codes 
of a Power H924 server. 


Table 2-3 Processor feature code specification for the Power H924 server 


Processor card Configuration Number of Typical Processor core 
feature code options cores frequency range | activation 
[GHz] feature code 


2.1.6 Memory access 


The Power H922 and Power H9?24 servers use direct-attached DDR4 industry-standard 
RDIMMs (IS RDIMMs). Each POWERS processor-based module has two memory controllers 
that operate four memory channels each. One memory channel can support up to two 
DIMMs, which implies a maximum of 16 DDR4 IS RDIMMs slots per POWER9 
processor-based module. 


If only one DIMM slot of a memory channel is populated, the configuration is referred to as 


single drop. If both DIMM slots of a memory channel are populated, the configuration is 
referred to as double-drop constellation. 


Chapter 2. Architecture and technical overview 55 


The nominal data rate of a memory RDIMM is different for single-drop and double-drop 
constellations. RDIMMs in single-drop mode that is run at 2666 or 2400 Mbps; in double-drop 
mode, the data rate is slightly lower at 2133 Mbps. 


Table 2-4 lists the RDIMM data rates for the available RDIMM sizes in single- (one DIMM per 
channel) and double- (two DIMMs per channel) drop mode. 


Table 2-4 RDIMM data rate for certain RDIMM sizes 


Data rate in single-drop mode | Data rate in double-drop mode 
(one DIMM per channel) (two DIMMs per channel) 


16 GB 2666 Mbps 2133 Mbps 


32 GB 2400 Mbps 2133 Mbps 
64 GB 2400 Mbps 2133 Mbps 
128 GB 2400 Mbps 2133 Mbps 


The Power H922 and Power H924 servers in a two-socket configuration can use up to 4 TB of 
memory in one processor configuration up to 2 TB memory. 


Figure 2-4 shows the POWERS direct-attached memory architecture that uses two memory 
control units (MCUs) with four DDR4 channels each. 


Figure 2-4 Overview of POWERS direct-attach memory architecture 


2.1.7 On-chip L3 cache innovation and intelligent caching 


Similar to POWERS, the POWERS processor uses a breakthrough in material engineering 
and microprocessor fabrication to implement the L3 cache in eDRAM and place it on the 
processor die. L3 cache is critical to a balanced design, as is the ability to provide good 
signaling between the L3 cache and other elements of the hierarchy, such as the L2 cache or 
SMP interconnect. 


The on-chip L3 cache is organized into separate areas with differing latency characteristics. 
Each processor core is associated with a fast, 10-MB local region of L3 cache (FLR-L3), but 
also can access other L3 cache regions as a shared L3 cache. 


Also, each core can negotiate to use the FLR-L3 cache that is associated with another core, 
depending on the reference patterns. Data can also be cloned and stored in more than one 
core’s FLR-L3 cache (again, depending on the reference patterns). This Intelligent Cache 
management enables the POWER39 processor to optimize the access to L3 cache lines and 
minimize overall cache latencies 
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The L3 cache on the POWERS processor includes the following features: 
> Private 10-MB L3 cache/shared L3 

> 20-way set associative 

> 128-byte cache lines with 64-byte sector support 

> 10EDRAM banks (interleaved for access overlapping) 

> 64-byte wide data bus to L2 for reads 

> 64-byte wide data bus from L2 for L2 castouts 


> 80 1-Mb EDRAM macros that are configured in 10 banks, with each bank having a 64-byte 
wide data bus 


> All cache accesses have the same latency 


> 20-way directory that is organized as four banks, with up to four reads, or two reads and 
two writes every two processor clock cycles to differing banks 


2.1.8 Hardware transactional memory 


Transactional memory is an alternative to lock-based synchronization. It attempts to simplify 
parallel programming by grouping read and write operations and running them as a single 
operation. 


Transactional memory is like database transactions where all shared memory accesses and 
their effects are committed together or discarded as a group. All threads can enter the critical 
region simultaneously. If conflicts exist in accessing the shared memory data, threads attempt 
to access the shared memory data again or are stopped without updating the shared memory 
data. Therefore, transactional memory is also called a lock-free synchronization. 
Transactional memory can be a competitive alternative to lock-based synchronization. 


Transactional memory provides a programming model that makes parallel programming 
easier. A programmer delimits regions of code that access shared data. The hardware runs 
these regions atomically and in isolation. It buffers the results of individual instructions and 
retries running if isolation is violated. Generally, transactional memory allows programs to use 
a programming style that is close to coarse-grained locking to achieve performance that is 
close to fine-grained locking. 


Most implementations of transactional memory are based on software. The POWER9 
processor-based systems provide a hardware-based implementation of transactional memory 
that is more efficient than the software implementations and requires no interaction with the 
processor core, which allows the system to operate at maximum performance. 


2.1.9 Coherent Accelerator Processor Interface 2.0 


IBM Coherent Accelerator Processor Interface (CAPI) 2.0 is the evolution of CAPI and defines 
a coherent accelerator interface structure for attaching special processing devices to the 
POWERS processor bus. As with the original CAPI, CAPI2 can attach accelerators that have 
coherent shared memory access with the processors in the server. CAPI2 shares full virtual 
address translation with these processors by using standard PCle Gen4 buses with twice the 
bandwidth compared to the previous generation. 


Applications can have customized functions in Field Programmable Gate Arrays (FPGAs) and 
queue work requests directly in shared memory queues to the FPGA. Applications can also 
have customized functions by using the same effective addresses (pointers) they use for any 
threads running on a host processor. 
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From a practical perspective, CAPI enables a specialized hardware accelerator to be seen as 
an extra processor in the system with access to the main system memory and coherent 
communication with other processors in the system. 


Figure 2-5 shows a comparison of the traditional model, where the accelerator must go 
through the processor to access memory with CAPI. 


Traditional Acceleration CAPI 


Device Driver 


Processor 


Processor 


I Memory 


Figure 2-5 CAPI accelerator that is attached to the POWER9 processor 


The benefits of the use of CAPI include the ability to access shared memory blocks directly 
from the accelerator, perform memory transfers directly between the accelerator and 
processor cache, and reduce the code path length between the adapter and the processors. 
This reduction in the code path length might occur because the adapter is not operating as a 
traditional I/O device, and no device driver layer is available to perform processing. CAPI also 
presents a simpler programming model. 


The accelerator adapter implements the POWER Service Layer, which provides address 
translation and system memory cache for the accelerator functions. The custom processors 
on the system board, consisting of an FPGA or an ASIC, use this layer to access shared 
memory regions and cache areas as though they were a processor in the system. This ability 
enhances the performance of the data access for the device and simplifies the programming 
effort to use the device. 


Instead of treating the hardware accelerator as an I/O device, it is treated as a processor, 
which eliminates the requirement of a device driver to perform communication. It also 
eliminates the need for direct memory access that requires system calls to the operating 
system kernel. By removing these layers, the data transfer operation requires fewer clock 
cycles in the processor, which improves the I/O performance. 


The implementation of CAPI on the POWER39 processor enables hardware companies to 
develop solutions for specific application demands. Companies use the performance of the 
POWERS processor for general applications and the custom acceleration of specific functions 
by using a hardware accelerator with a simplified programming model and efficient 
communication with the processor and memory resources. 
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2.1.10 Power management and system performance 


The POWER39 processor-based Power H922, and Power H924 servers implement Workload 
Optimized Frequency (WOF) technology as a feature of the IBM EnergyScale power 
management technology. With IBM EnergyScale in a POWERS9 processor-based server, the 
POWERS processor-based dynamic power saver (DPS) modes that favor lower power 
consumption or favor performance are replaced by two new power saver modes: 


> Dynamic performance mode (DPM) 
> Maximum performance mode (MPM) 


Every POWER9 processor-based has DPM or MPM enabled by default. For the Power H922 
(9223-22S) and Power H924 (9223-42S), servers have MPM enabled by default. 


DPM and MPM dynamically adjust the processor frequency to maximize performance. Each 
of these power saver modes delivers consistent system performance without any variation if 
the nominal operating environment limits are met. 


For POWERS processor-based servers that are under control of the PowerVM hypervisor, the 
DPM and MPM are system-wide configuration settings, but each processor module frequency 
is optimized separately. 


Several factors determine the maximum frequency that a processor module can run at: 
> Processor utilization: Lighter workloads run at higher frequencies. 
> Number of active cores: Fewer active cores run at higher frequencies. 


> Environmental conditions: At lower ambient temperatures, cores are enabled to run at 
higher frequencies. 


The new power saver modes are defined as follows: 


DPM In DPM, the workload is run at the highest frequency possible if the 
nominal power consumption limit of the processor modules is 
accepted. The frequency of the processor modules is always at the 
nominal frequency of the POWER9 processor-based server or above 
the nominal frequency up to the upper limit of the DPM frequency 
range. 


This DPM typical frequency range (DTFR) is published as part of the 
system specifications of a particular POWER9 processor-based 
server if it is running by default in the DPM. The system performance 
is deterministic within the allowed operating environmental limits, so it 
does not depend on the ambient temperature if the temperature stays 
within the supported range. 


The idle power saver (IPS) function can be enabled or disabled. If IPS 
is enabled and all cores in a processor module are idle for hundreds of 
milliseconds, the frequency of the cores in the respective module drop 
to the predefined power save frequency. 


MPM In MPM, the workload is run at the highest frequency possible, but 
unlike in the DPM, the processor module can operate at a higher 
power consumption level. The higher power draw enables the 
processor modules to run in an MPM typical frequency range (MTFR) 
where the lower limit is above the nominal frequency and the upper 
limit is determined by the system’s maximum frequency. 


The MTFR is published as part of the system specifications of a 
specific POWERS processor-based if it is running by default in MPM. 
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The higher power draw potentially increases the fan speed of the 
respective system node to meet the higher cooling requirements, 
which cause a higher noise emission level of up to 15 decibels. The 
processor frequency typically stays within the limits that are provided 
by the MTFR, but can be lowered to frequencies between the MTFR 
lower limit and the nominal frequency at high ambient temperatures 
above 25 °C (77 °F). If the data center ambient environment is less 
than 25 °C, the frequency in MPM is consistently in the upper range of 
the MTFR (roughly 10% - 20% better than nominal). At lower ambient 
temperatures (below 25 °C), MPM mode also provides deterministic 
performance. As the ambient temperature increases above 25 °C, 
determinism can no longer be guaranteed. 


The IPS function can be enabled or disabled. If IPS is enabled, the 
frequency is dropped to the static power saver level if the entire 
system meets the configured idle conditions. 


Figure 2-6 shows the frequency ranges for the POWER3 static nominal mode (all modes 
disabled), the DPM, and the MPM. The frequency adjustments for different workload 
characteristics, ambient conditions, and idle states are also indicated. 
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Figure 2-6 POWER9 power management modes and related frequency ranges 
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Table 2-5 and Table 2-6 list the static nominal frequencies and the frequency ranges of the 
DPM and the MPM of all processor module types that are available for Power H922, and 
Power H924 servers. 


Table 2-5 Characteristic frequencies and frequency ranges for Power H922 server 


Feature Cores per Static nominal | DPM frequency range MPM frequency range 
code SCM frequency [GHz] [GHz] 
[GHz] 


2.3 : : 
: 


Table 2-6 Characteristic frequencies and frequency ranges for Power H924 server 


Feature Cores per Static nominal | DPM frequency range MPM frequency range 
code SCM frequency [GHz] [GHz] 
[GHz] 


#EP7M 2.817 2.817 - 3.9 3.45 - 3.9 


For more information about IBM EnergyScale, see /BM EnergyScale for POWER9 
Processor-Based Systems. 


2.2 Memory subsystem 


The POWER9Y MCU provides the system memory interface between the on-chip SMP fabric 
and the memory interface physical unit (DDR PHY unit), which connects to DDR4 
industry-standard RDIMMs (IS RDIMMs). 


Logically, eight independent MCUs are on the POWER39 processor that interface to eight 
DDR4/4E memory buses. Each MCU memory port supports up to two DDR4 DIMM slots. If 
only one DIMM slot is populated, the configuration is referred to as a single-drop 
constellation; if both DIMM slots are populated, the configuration is referred to as a 
double-drop constellation. 


Physically, the MCUs are grouped into two instances of the extended memory controller 
(EMC) chiplet: MCU Group 0 and MCU Group 1. Therefore, each EMC chiplet contains four 
MCUs plus pervasive logic. 


On new systems orders, IS RDIMMs with 16 GB, 32 GB, 64 GB, and 128 GB are used in 
various combinations throughout the new Power H922 (9223-22S) and Power H924 
(9223-42S) server family. 


These servers are two-socket servers that support one or two POWER9 processor-based 


modules. The servers offer a maximum of 32 DDR4 DIMM slots, with 16 DIMM slots per 
installed processor module. 
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Memory features for 16 GB (#EM6J), 32 GB (#EM6kK), 64 GB (#EM7B), and 128 GB 
(#EM7C) are available for all processor module configurations of the Power H922 and 
Power H924 systems at initial order. 


By using the 128 GB RDIMM size on all 32 DDR4 DIMM slots, the servers can have a 
maximum of 4 TB system memory. Memory features must be ordered in pairs and at least two 
memory slots must be populated per processor module, which implies a minimum memory 
capacity of 32 GB for one-socket and 64 GB for two-socket configurations. Memory data rates 
vary depending on the RDIMM size and module placement, as shown in Table 2-4 on 

page 56. 


The maximum theoretical memory bandwidth for a POWER9 processor-based module is 
170 GBps. The total maximum theoretical memory bandwidth for a two-socket system is 
340 GBps. 


These servers support an optional feature that is called Active Memory Expansion that 
enables the effective maximum memory capacity to be much larger than the true physical 
memory on AIX operating system. This feature runs innovative compression and 
decompression of memory content by using a dedicated coprocessor to provide memory 
expansion up to 125%, depending on the workload type and its memory usage. 


For example, a server with 256 GB RAM physically installed can effectively be expanded over 
512 GB RAM. This approach can enhance virtualization and server consolidation by allowing 
a partition to do more work with the same physical amount of memory or a server to run more 
partitions and do more work with the same physical amount of memory. 


2.2.1 Memory placement rules 


The Table 2-7 lists the available to order memory options for the Power H922 and 
Power H924 servers. 


Table 2-7 Memory DIMM options for Power H922 and Power H924 servers 


Memory DIMM feature code Memory DIMM size 
#EM6J 16 GB DDR4 DRAM 


#EM6K 32 GB DDR4 DRAM 
#EM7B 64 GB DDR4 DRAM 
#EM7C 128 GB DDR4 DRAM 


The minimum memory capacity for the Power H922 and Power H924 servers is 32 GB for 
one-socket (2 x #EM6J), and 64 GB for two-socket (4 x #EM6J) configurations, because at 
least two memory slots must be populated per processor module. 


The supported maximum memory is as follows for the Power H922 and the Power H924 
servers: 

> One processor module installed: 2 TB (16 128 GB RDIMMs) 

> Two processors modules installed: 4 TB (32 128 GB RDIMMs) 

Consider the following basic rules for memory placement: 

>» Each feature code equates to a single physical RDIMM. 


> All memory features must be ordered in pairs. 
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> All memory features must be ordered in even quantities: 


— For single-processor module configurations, the supported quantities of RDIMMs are 
2, 4, 6, 8, 12, and 16. Quantities of 10 or 14 RDIMMs are not supported. 


— For double-processor module configurations, the supported quantities of RDIMMs are 
4, 6, 8, 10, 12, 14, 16, 20, 24, 28, and 32. Quantities of 18, 22, 26, or 30 RDIMMs are 
not supported. 


If the configuration rules require you to order a quad of memory features, not all features 
must be the same size. Instead, they can be a combination of two memory feature pairs of 
different sizes. 


> All memory RDIMMs must be installed in pairs. 


> Each RDIMM within a pair must be of the same capacity. 


In general, the preferred approach is to install memory evenly across all processors in the 
system. Balancing memory across the installed processors enables memory access in a 
consistent manner and typically results in the best possible performance for your 
configuration. Account for any plans for future memory upgrades when you decide which 
memory feature size to use at the time of the initial system order. 


One socket configuration 


Figure 2-7 shows the physical memory DIMM slot topology for one-socket configurations of 
the Power H922 and Power H924 servers. 
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Figure 2-7 Memory DIMM slot topology for one-socket configurations 
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For servers with a single processor module that is installed, the plugging order for the 
memory RDIMMs is as follows (see Figure 2-8): 


> Pair installation: 
The first RDIMM pair is installed at Red 1 (C33 DDRO-A and C17 DDR1-A). 
— The second RDIMM pair is installed at Gold 2 (C36 DDR4-A and C22 DDR5-A). 
The third RDIMM pair is installed at Cyan 3 (C31 DDR2-A and C15 DDR3-A). 

— The fourth RDIMM pair is installed at Gray 4 (C38 DDR6-A and C20 DDR7-A). 
> Quad installation: 

— Two fifth RDIMM pairs (or quad) are installed at Red 5 (C34 DDRO-B and C18 

DDR1-B) and at Cyan 5 (C32 DDR2-B and C16 DDR3-B). 


— Two sixth RDIMM pairs (or quad) are installed at Gold 6 (C35 DDR4-B and C21 
DDR-B) and at Gray 6 (C37 DDR6-B and C19 DDR7-B). 


MCU Group 0 MCU Group 1 
DDRO DDR1 DDR2 DDR3 DDR4 DDRS DDR6 DDR7 
A B A B A B A B A B A B A B A B 
C33 C34 C17 C18 C31 C32 (Gils C16 C36 C35 C22 C21 C38 (cey/ C20 c19 
a a a 
| Sy 6| 6| 


Figure 2-8 RDIMM plug sequence for one-socket configurations 


Consider the following points: 


> You cannot mix RDIMMs of different capacity and ranking in a single-drop constellation 
within an MCU group. Table 2-8 lists the size and the rank count of all supported memory 
feature codes for Power H922, and Power H924 systems. 


Table 2-8 Size and rank type of supported memory feature codes 


[reawocese [eR 


a. 1R designates RDIMMs with one rank and 2R designates RDIMMs with two ranks. 


> The memory feature order and placement rules require that the transition from a 
single-drop to double-drop constellation occurs on MCU group boundaries. The 
installation of a quad transforms the respective MCU group from a single-drop to a 
double-drop constellation. 


> RDIMMs in the same color cells must be identical (same size and rank). 


Note: Each color in Figure 2-8 represents a unique RDIMM size and type. 
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Two socket configuration 


Figure 2-9 shows the physical memory DIMM topology for two-socket configurations of the 
Power H922 and Power H924 servers. 
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Figure 2-9 Memory DIMM slot topology for two socket configuration 
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For servers with dual processor modules that are installed, the plugging order for the memory 
RDIMMs is as follows (see Figure 2-10): 


> Pair installation: 
— The first DIMM pair is installed at Red 1 (C33 DDRO-A and C17 DDR1-A) of SCM-O. 


— The second DIMM pair is installed at Green 2 (C41 DDRO-A and C25 DDR1-A) of 
SCM-1. 


— The third DIMM pair is installed at Gold 3 (C36 DDR4-A and C22 DDR5-A) of SCM-O. 


— The fourth DIMM pair is installed at Purple 4 (C44 DDR4-A and C30 DDR5-A) of 
SCM-1. 


— The fifth DIMM pair is installed at Cyan 5 (C31 DDR2-A and C15 DDR3-A) of SCM-O. 
— The sixth DIMM pair is installed at Pink 6 (C39 DDR2-A and C23 DDR3-A) of SCM-1. 


— The seventh DIMM pair is installed at Gray 7 (C38 DDR6-A and C20 DDR7-A) of 
SCM-O. 


— The eighth DIMM pair is installed at Yellow 8 (C46 DDR6-A and C28 DDR7-A) of 
SCM-1. 


> Quad installation: 


— Two ninth DIMM pairs (or quad) are installed at Red 9 (C34 DDRO-B and C18 DDR1-B) 
and at Cyan 9 (C32DDR2-B and C16 DDR3-B) of SCM-O. 


— Two 10th DIMM pairs (or quad) are installed at Green 10 (C42 DDRO-B and C26 
DDR1-B) and at Pink 10 (C40 DDR2-B and C24 DDR3-B) of SCM-1. 


— Two 11th DIMM pairs (or quad) are installed at Gold 11 (C35 DDR4-B and C21 
DDR5-B) and at Gray 11 (C37 DDR6-B and C19 DDR7-B) of SCM-O. 


— Two 12th DIMM pairs (or quad) are installed at Purple 12 (C43 DDR4-B and C29 
DDR5-B) and at Yellow 12 (C45 DDR6-B and C27 DDR7-B) of SCM-1. 


MCU Group 0 MCU Group 1 MCU Group 0 MCU Group 1 
DDRO | DDR1 | DDR2 | DDR3 | DDR4 | DDRS | DDR6 | DDR7 | DDRO | DDR1 | DDR2 | DDR3 | DDR4 | DDRS | DDR6 | DDR7 
A_{B A |B JA {[B A_{[B A |B {A |B 
C31 |C32 C22 }C21 |C38 |C37 C25 |C26 C30 |C29|C46 |C45 


Figure 2-10 RDIMM plug sequence for two socket configuration 


Consider the following points: 


> You cannot mix RDIMMs of different capacity and ranking in a single-drop constellation 
within an MCU group. A two-socket Power H922 or Power H924 server runs in single-drop 
constellation if it contains 16 or fewer memory modules; therefore, the size and rank type 
of all RDIMMs within each MCU group must be identical. Table 2-4 on page 56 lists the 
size and the rank count of all supported memory feature codes for Power H922, and 
Power H924 servers. 


> DIMMs at same color cells must be identical (Same size and rank). 
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2.2.2 Memory bandwidth 


The Power H922 and Power H924 servers can be configured with one or two POWER9 
processor modules. Each processor module drives eight memory channels that are 
organized in two MCU groups. Each memory channel or port supports single- or double-drop 
industry-standard RDIMMs, so each channel can accommodate one or two DIMMs. 


As data flows from main memory toward the execution units of the POWER9 processor cores, 
they pass through the 512 KB L2 and the 64 KB L1 cache. In many cases, the 10 MB L3 
victim cache might also provide the data that is needed for the instruction execution. 


The following sections describe the cache and memory bandwidth capabilities of the 
Power H922 and Power H924 servers. 


Power H922 (9223-22S) cache and memory bandwidth 


Table 2-9 lists the maximum cache bandwidth for a single core as defined by the width of the 
relevant channels and the related transaction rates of a Power H922 server. 


Table 2-9 Power H922 (9223-22S) single core architectural maximum cache bandwidth 


cen 


The bandwidth figures for the caches are calculated as follows: 


> Li data cache: In one clock cycle, four 16-byte load operations and two 16-byte store 
operations can be accomplished. The values vary depending on the core frequency and 
are computed as follows: 


— Core running at 3.8 GHz: (4 x 16 B+ 2 x 16 B) x 3.8 GHz = 364.8 GBps 
— Core running at 3.9 GHz: (4 x 16 B+ 2x 16 B) x 3.9 GHz = 374.4 GBps 


>» L2 cache: In one clock cycle, one 64-byte read operation to the core and two 16-byte store 
operations from the core can be accomplished. The values vary depending on the core 
frequency and are computed as follows: 


— Core running at 3.8 GHz: (4 x 16 B+ 2 x 16 B) x 3.8 GHz = 364.8 GBps 
— Core running at 3.9 GHz: (4 x 16 B+ 2 x 16 B) x 3.9 GHz = 374.4 GBps 


> L3 cache: With two clock cycles, one 64-byte read operation to the L2 cache and one 
64-byte store operation from the L2 cache can be accomplished. The values vary 
depending on the core frequency and are computed as follows: 


— Core running at 3.8 GHz: (1 x 64 B+ 1 x 64 B) x 3.8 GHz / 2 = 243.2 GBps 
— Core running at 3.9 GHz: (1 x 64 B+ 1 x 64 B) x 3.9 GHz / 2 = 249.6 GBps 
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For the maximum configuration with two processor modules that are installed and all memory 
channels that are populated in single-drop constellation, the overall aggregated memory 
bandwidth figures of the Power H922 server as defined by the width of the relevant channels 
and the related transaction rates are listed in Table 2-10. 


Table 2-10 Power H922 (9223-22S) total architectural maximum cache and memory bandwidth 


20 cores @ 3.8 GHz 
(maximum) (maximum) 


Power H924 (9223-42S) cache and memory bandwidth 
Table 2-11 lists the maximum cache bandwidth for a single core as defined by the width of the 
relevant channels and the related transaction rates of a Power H924 server. 


Table 2-11 Power H924 (9223-42S) single core architectural maximum cache bandwidth 


Single core Power H924 server 


1 core @ 3.9 GHz (maximum) | 1 core @ 4.0 GHz (maximum) 


L1 data cache 374.4 GBps 384 GBps 


The bandwidth figures for the caches are calculated as follows: 


> L1 data cache: In one clock cycle, four 16-byte load operations and two 16-byte store 
operations can be accomplished. The values vary depending on the core frequency and 
are computed as follows: 


— 3.9 GHz core: (4x 16 B + 2x 16 B) x 3.9 GHz = 374.4 GBps 
— 4.0 GHz core: (4x 16 B +2 x 16 B) x 4.0 GHz = 384 GBps 


>» L2 cache: In one clock cycle, one 64-byte load operation and two 16-byte store operations 
can be accomplished. The values vary depending on the core frequency and are 
computed as follows: 


— 3.9 GHz core: (4x 16 B + 2x 16 B) x 3.9 GHz = 374.4 GBps 
— 4.0 GHz core: (4x 16 B +2 x 16 B) x 4.0 GHz = 384 GBps 


> L3 cache: With two clock cycles, one 64-byte read operation to the L2 cache and one 
64-byte store operation from the L2 cache can be accomplished. The value depends on 
the core frequency and is computed as follows: 
— Core running at 3.9 GHz: (1 x 64 B+ 1 x 64 B) x 3.9 GHz / 2 = 249.6 GBps 
— Core running at 4.0 GHz: (1 x 64 B+ 1 x 64 B) x 4.0 GHz / 2 = 256 GBps 
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For the maximum configuration with two processor modules that are installed and all memory 
channels that are populated in single-drop constellation, the overall aggregated memory 
bandwidth figures of the Power H924 server as defined by the width of the relevant channels 
and the related transaction rates are listed in Table 2-12. 


Table 2-12 Power H924 (9223-42S) total architectural maximum cache and memory bandwidth 


Total bandwidths Power H924 server 


16 cores @ 4.0 GHz 20 cores @ 3.9 GHz 24 cores @ 3.9 GHz 
(maximum) (maximum) (maximum) 
L1 (data) cache 6144 GBps 7488 GBps 8985.6 GBps 
6144 GBps 7488 GBps 8985.6 GBps 


Note: Several POWERS design points must be considered when comparing hardware 
designs that use SMP communication bandwidths as a unique measurement. The 
POWER3 architecture provides the following benefits: 


> More cores per socket, which leads to lower inter-CPU communication. 


> More RAM density (up to 2 TB per socket), which leads to less inter-CPU 
communication. 


> Greater RAM bandwidth for less dependence on an L3 cache. 
> Intelligent hypervisor scheduling that places RAM usage close to the CPU. 
> New SMP routing so that multiple channels are available when congestion occurs. 


Memory bandwidth considerations 

The Power H922, and Power H924 servers are memory performance-optimized with eight 
DIMM slots that are populated per processor module. In this configuration, all memory DIMMs 
work in single-drop mode with a data rate of 2666 or 2400 Mbps. If both DIMMs are populated 
(double drop) in a memory channel, the data rate is reduced to 2133 Mbps. 


The maximum bandwidth of the system depends on the DIMM sizes, the number of channels 
that are used, and whether the channel operates in single-drop or double-drop mode. 


To calculate the maximum bandwidth, use the following formula: 


Maximum Bandwidth = Number of DIMMs in single-drop mode x 21.25 GBps) + ((Number 
of DIMMs in double-drop mode / 2) x 17.06 GBps) 
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Table 2-13 lists the maximum memory bandwidth of the Power H922 and Power H924 
servers, depending on the number of DIMMs that are populated. 


Table 2-13. Maximum memory bandwidth of Power H922 (9223-22S) and Power H924 (9223-42S) 


DIMM quantity Maximum bandwidth in GBps? 


a. Numbers are rounded to the nearest integer. 

b. MCU Group 0 of POWERY SCM 0 operates in double-drop mode. 

c. MCU Group 0 of POWER9 SCM 0 and SCM 1 operate in double-drop mode. 

d. MCU Group 0 and MCU Group 1 of POWER9 SCM 0 and MCU Group 0 of SCM 1 operate in 
double-drop mode. 

e. MCU Group 0 and MCU Group 1 of POWER9 SCM 0 and SCM 1 operate in double-drop mode. 


2.3 System buses 


The Power H922 and Power H924 servers are based on the POWER9 SCM in a package that 
is optimized for commercial entry servers. For each of the named systems, one processor 
module supports the following buses: 


> Eight DDR4 ports support a maximum 2666 Mbps data rate if a single DIMM per channel 
is used and a maximum 2133 Mbps data rate if two DIMMs per channel are used. 


> Two 30-bit + two spare (4-byte) inter-node SMP X buses at 16 Gbps signaling rate. 
> 42 lanes PCle Gen 4 at 16 Gbps signaling rate. 
>» Processor service interface (PSI) bus. 


These buses are described next. For more information, see Figure 2-1 on page 48 and 
Figure 2-2 on page 49. 


2.3.1 Memory buses 


Logically, eight independent MCUs are implemented on a POWERY SCM. Each MCU 
provides the interface to one DDR4 memory bus and each of the related MCU memory ports 
provide access to two DDR4 DIMM slots. IS RDIMMs with DRAM data rates of 2133, 2400, 
and 2666 Mbps are supported to populate the DIMM slots. 
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For more information about the memory subsystem architecture, the available memory 
features and related placement rules, and the relevant bandwidth figures, see 2.2, “Memory 
subsystem” on page 61. 


2.3.2 Inter-node SMP X-buses 


Each POWER9 SCM provides two X-bus links that are used in 2-socket Power H922 and 
Power H924 servers to create a scalable cache-coherent SMP system. The differential X-bus 
contains two clock groups. Each clock group consists of one clock, 16 data lanes, and one 
spare data lane. Both clock groups are connected to the same processor. 


The X-bus runs at a rate of 16 GHz. The peak bandwidth is 60 GBps per link with a peak data 
bandwidth of 48 GBps because of cyclic redundancy check (CRC) logic. 


The two X-bus inter-node SMP connections between the sockets in Power H922 or 
Power H924 servers deliver a peak data bandwidth of 96 GBps. 


2.3.3 PCle Gen 4 buses 


In Power H922 and Power H924 servers, each POWERQY SCM integrates three PCle Gen 4 
controllers: EO, E1, and E2. EO and E2 have 16 lanes and E1 has 10 lanes for a total of 42 
lanes with a full duplex bandwidth of 4 GBps per lane. The 42 lanes can be configured to 
support up to six independent PCle host bridges (PHBs) or PCle buses. The number of used 
PHBs of an SCM is variable and depends on the position of the socket on the system board 
(SCMO or SCM1). 


Table 2-14 lists the PCle bus configuration for the first socket (SCMO) of a Power H922 ora 
Power H924 server. 


Table 2-14 PCle Gen 4 I/O configuration for POWER9 SCMO 


PCleGen4 | PClehost | Number Slot connection | Slot number and specification? 
controller bridge of lanes type 
PHBO C9 PCle4 x16 CAP! 2.0 capable LP/FHHL 


EY Multiplexed C10 PCle4 x8 LP/FHHL 
through 52-lane 
PCle Gen 4 C11 PCle4 x8 LP/FHHL 


switch C12 PCle4 x16 LP/FHHL 


C49 PCle4 x8 with x16 connector internal slot: 
internal SAS adapter or NVMe pass-through card 


Ple2 USB 8:0 4 port 


Multiplexed C5 PCle4 x8 LP/FHHL 


through 52-lane 
PCle Gen 4 C6 PCle4 x16 LP/FHHL 


switch 


C7 PCle4 x8 LP/FHHL 


C50 PCle4 x8 with x16 connector internal slot: 
internal SAS adapter or NVMe pass-through card 


C8 x8 CAPI 2.0 capable with x16 connector LP/FHHL 


a. The Power H922 server supports only low-profile (LP) adapters, and Power H924 servers support only full-height 
half-length (FHHL) adapters. 
b. The PCle host bridge PHB2 supports x2 lanes, but only one lane is needed to back the USB interface. 
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c. Two USB ports are accessible from the front and two USB ports at the rear of a Power H922 server, and 
Power H924 servers provide 1 USB port at the front and 2 USB ports at the rear. 


The internal slots C49 and C50 can each accommodate #EJ1Q NVMe U.2 pass-through 
cards. One #EJ1Q card is automatically added to the system if feature #EJ1V storage 
backplane Gen 4 with two NVMe U.2 drive slots is configured. Two #EJ1Q cards are 
automatically added to the system if feature #EJ1W storage backplane Gen 4 with four NVMe 
U.2 drive slots are chosen. 


The #EJ1Q NVMe U.2 pass-through card splits the eight incoming lanes from the internal 
PCle Gen 4 switch into two bundles of four lanes that are used to support one independent 
NVMe U.2 form factor drive per 4-lane bus. 


The total aggregated PCle Gen 4 I/O full duplex bandwidth that is available for a one-socket 
system adds up to 160 GBps, of which 96 GBps is directly provided to two external PCle slots 
(C8 and C9). A total of 64 GBps is available through the six external PCle slots (C5, C6, C7, 
C10, C11, and C12) that are attached to the two internal PCle Gen 4 switches. 


Table 2-15 lists the PCle bus configuration for the second socket (SCM1) of Power H922 or 
Power H924 2-socket configurations. 


Table 2-15 PCle Gen 4 I/O configuration for POWER9 SCM1 in two-socket configurations 


PCleGen4 | PClehost | Number Slot connection | Slot number and specification? 
controller eee of lanes type 


PHBO C4 PCle4 x16 CAPI 2.0 capable LP/FHHL 


=— PHB1 a eee C2 PCle4 x8 with x16 connector LP/FHHL 
PHB3 C3 PCle4 x16 CAPI 2.0 capable LP/FHHL 


a. The Power H922 server supports only LP adapters, and the Power H924 server supports only FHHL adapters. 


The total aggregated PCle Gen 4 I/O full duplex bandwidth that is available for a two-socket 
server adds up to 320 GBps, of which 256 GBps is directly provided to five external PCle 
slots (C2, C3, C4, C8, and C9). A total of 64 GBps is available through the six external PCle 
slots (C5, C6, C7, C10, C11, and C12) that are attached to the internal PCle Gen 4 switches. 


Table 2-16 lists the aggregated full duplex PCle Gen 4 bandwidth that is available through the 
integrated internal and external PCle adapter slots. 


Table 2-16 PCle Gen 4 full duplex bandwidth for Power H922, and Power H924 servers 


PCle Gen 4 full duplex bandwidth Power H922 Power H924 
One-socket system total 160 GBps 160 GBps 


| Two-socket systemtotal = socket | Two-socket systemtotal = total |s20GBps |s20GBps |s20GBps |s20GBps 


Aggregated over direct-attached slots ofa | 96 GBps 96 GBps 
one-socket system 


Aggregated over direct-attached slots ofa | 256 GBps 256 GBps 
two-socket system 

Aggregated over slots that are attachedto | 64 GBps 64 GBps 
the PCle Gen 4 switches 
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2.3.4 Service processor interface bus 


Power H922 and Power H924 servers are always delivered with one Flexible Service 
Processor (FSP) card in slot P1-C1 at the rear of the system. The FSP is required to conduct 
system maintenance and virtualization configuration tasks. The processor service interface 
(PSI) bus that is implemented on POWERS processor-based modules is used to connect 
each installed SCM to the FSP card. 


2.4 Internal I/O subsystem 


As described in 2.3, “System buses” on page 70, each POWER®Y SCM in a Power H922 or 
Power H924 server integrates three PCle Gen 4 controllers for a total of 42 lanes. Depending 
on the specific system configuration, the PCle controllers connect to 8 PCle slots (a 
one-socket system) or 11 PCle slots (a two-socket system). 


All PCle slots are implemented in PCle Gen 4 technology and support hot-plug adapter 
installation and maintenance and enhanced error handling (EEH). PCle EEH-enabled 
adapters respond to a special data packet that is generated from the affected PCle slot 
hardware by calling system firmware, which examines the affected bus, allows the device 
driver to reset it, and continues without a system restart. 


PCle slots that are directly connected to a POWER9 SCM show the best performance 
characteristics and are be used to install high-performance adapters. This recommendation is 
reflected and further elaborated by the I/O adapter enlarged capacity enablement order that is 
listed in Table 2-18 on page 74, Table 2-19 on page 75, Table 2-20 on page 75, and the PCle 
adapter placement documents for the Power H922 (9223-22S) and Power H924 (9223-42S) 
servers in IBM Knowledge Center. 


The POWER9 SCMs that are connected directly to the high-performance slots also support 
CAPI 2.0 adapters for the most demanding I/O channels. Per installed socket, two 
high-performance slots are CAPI 2.0 capable and can be populated (for example) with 

100 Gb InfiniBand Enhanced Data Rate (EDR) CAPI adapters (#EC62 and #EC63). 


All PCle Gen 4 adapter slots of the Power H922 and Power H924 servers support 
hardware-backed network virtualization through single root IO virtualization (SR-IOV) 
technology. 


2.4.1 Slot configuration 


The various slot configurations are described in this section. For each server model, the PCle 
slot locations are listed by the slot type and the CAPI capability. The I/O adapter enlarged 
capacity enablement order is provided for each PCle adapter slot. Figures help you find the 
PCle slots locations at the rear of the servers. 


Slot configuration for the Power H922 server 

The number of PCle slots that are available with a Power H922 server depends on the 
number of installed processor modules. Table 2-17 on page 74 lists the available PCle slot 
types and the related slot location codes in Power H922 servers. The 2U height of a 
Power H922 server allows only for the installation of PCle adapters in the half-height 
half-length (HHHL) LP form factor. 
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Table 2-17 PCle slot locations for a slot type in the Power H922 server 


Slot type Numberof | Location codes 
slots 


PCle Gen 4 x8 P1-C5, P1-C7, P1-C10, and P1-C11 


PCle Gen 4 x8 with x16 connector P1-C2 and P1-C8 
5 


PCle Gen 4 x16 P1-C39, P1-C42, P1-C6, P1-C9, and 
P1-C12 


a. This slot is available when the second processor module is populated. 


Table 2-18 lists the PCle adapter slot locations and related characteristics for the Power H922 
server. 


Table 2-18 PCle slot locations and capabilities for the Power H922 server 


Location | Description Parent device CAPI 2.0 /O adapter 

code capability | enlarged 
capacity 
enablement 
order? 


P1-C2° PCle Gen 4 x8 with x16 connector | Processor SCM1 
P1-C3> PCle Gen 4 x16 Processor SCM1 
P1-c4 PCle Gen 4 x16 Processor SCM1 


P1-C5 PCle Gen 4 x8 PCle Gen 4 switch 
P1-C6 


P1-C11 PCle Gen 4 x8 (default local area 
network (LAN) slot) 


P1-C12 PCle Gen 4 x16 


a. Enabling the I/O adapter enlarged capacity option affects only Linux partitions. 
b. A high-performance slot that is directly connected to the processor module. 


Figure 2-11 shows the rear view of the Power H922 server with the location codes for the 
PCle adapter slots. The PCle slot C5 is populated with an optional USB 3.0 adapter in the 
figure. 


Figure 2-11 Rear view of a rack-mounted Power H922 server with PCle slots location codes 
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Slot configuration for Power H924 server 

The number of PCle slots that are available with a Power H924 server depends on the 
number of installed processor modules. Power H924 servers are available in one-socket or 
two-socket configurations. 


Table 2-19 lists the available PCle slot types and the related slot location codes for 
Power H924 servers. 


Table 2-19 PCle slot locations for a slot type in Power H924 servers 


Slot type Numberof | Location codes 
slots 


PCle Gen 4 x8 a P1-C5, P1-C7, P1-C10, and P1-C11 


} PCle Gen 4 x8 with x16 connector —_| Gen 4 x8 with x16 connector p20 | Pt-c28 and P1-C8 


occu ieee | Gen 4 x16 2 free P1-C4?, P1-C6, P1-C9, and 
P1-C12 


a. This slot is available when the second processor module is populated. 


Table 2-20 lists the PCle adapter slot locations and related characteristics for the same type 
of servers. 


Table 2-20 PCle slot locations and capabilities for the Power H924 server 


Location | Description Parent device CAPI 2.0 /O adapter 

code capability | enlarged 
capacity 
enablement 
order? 


[ries [rowan _—[ Powcnaemen we fe _ 
Ce 


a. Enabling the I/O adapter enlarged capacity option affects only Linux partitions. 
b. A high-performance slot that is directly connected to the processor module. 
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Figure 2-12 shows the rear view of the Power H924 server with the location codes for the 
PCle adapter slots. 
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Figure 2-12 Rear view of a rack-mounted Power H924 server with the PCle slots location codes 


2.4.2 System ports 


The system board has one serial port that is called a system port. The one system port is 
RJ45 and supported by AIX and Linux for attaching serial devices, such as an asynchronous 
device, for example, a console. If the device does not have a RJ45 connection, a converter 
cable, such as #3930, can provide a 9-pin D-shell connection. 


2.5 PCle adapters 


This section covers the various types and functions of the PCI adapters that are supported by 
the Power H922 (9223-22S) and Power H924 (9223-42S) servers. 


Important: No Fibre Channel over Ethernet (FCoE) support ion POWERQ 
processor-based servers is available. 


2.5.1 Peripheral Component Interconnect Express 


PCle uses a serial interface and enables point-to-point interconnections between devices (by 
using a directly wired interface between these connection points). A single PCle serial link is 
a dual-simplex connection that uses two pairs of wires (one pair for transmit and one pair for 
receive) that can transmit only 1 bit per cycle. These two pairs of wires are called a lane. A 
PCle link can consist of multiple lanes. In such configurations, the connection is labeled as 
x1, x2, x8, x12, x16, or x32, where the number is effectively the number of lanes. 
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The PCle interfaces that are supported on this server are PCle Gen 4, which are capable of 
32 GBps simplex (64 GBps duplex) on a single x16 interface. PCle Gen 4 slots also support 
previous generations (Gen3, Gen2, and Gen1) adapters, which operate at lower speeds, 
according to the following rules: 


> Place x1, x4, x8, and x16 speed adapters in the same size connector slots before mixing 
the adapter speed with the connector slot size. 


> Adapters with smaller speeds are allowed in larger-sized PCle connectors, but 
higher-speed adapters are not compatible in smaller connector sizes (that is, a x16 
adapter cannot go into an x8 PCle slot connector). 


All adapters support EEH. PCle adapters use a different type of slot than PCI adapters. If you 
attempt to force an adapter into the wrong type of slot, you might damage the adapter or the 
slot. 

POWERS processor-based servers can support two different form factors of PCle adapters: 


> PCle LP cards, which are used with the Power H922 PCle slots. These cards are not 
compatible with Power H924 servers because of their low height, but similar cards are 
used in other form factors. 


> PCle full-height and full-high cards are not compatible with the Power H922 server and are 
designed for Power H924 server. 


Before adding or rearranging adapters, use the IBM System Planning Tool (SPT) to validate 
the new adapter configuration. 


If you are installing a new feature, ensure that you have the software that is required to 
support the new feature and determine whether any update prerequisites must be met to 
install by seeing the IBM Power Systems Prerequisite website. 


The following sections describe the supported adapters and provide tables of orderable 


feature numbers. The tables indicate operating system support (AIX, IBM i, and Linux) for 
each of the adapters. 


Note: The maximum number of adapters in each case might require the server to have an 
external I/O drawer. 


2.5.2 LAN adapters 


To connect the Power H922 and Power H924 servers to a LAN, you can use the LAN 
adapters that are supported in the PCle slots of the system unit. 


Table 2-21 lists the LAN adapters that are available for the Power H922 (9223-22S) server. 


Table 2-21 Available LAN adapters for the Power H922 (9223-22S) server 


Feature CCIN Description OS support 
code 


5260 576F | PCle2 LP 4-port 1 GbE Adapter 4 AIX, IBM i, and Linux 


} 5809 | s76F | | PCle2 4-port1 GbE Adapter | 4-port 1 GbE | PCle2 4-port1 GbE Adapter | AIX, IBMi, | AIX, IBMi, and Linux | Linux 


we = PCle3 LP 2-Port 10 Gb NIC&ROCE SR/Cu Ee IBM i, and Linux 
Adapter 

58FA | PCle3 2-Port 10 Gb NIC&ROCE SR/Cu —— AIX, IBMi, and Linux 
Adapter 
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Feature CCIN Description OS support 

code 

EC2T 58FB PCle3 LP 2-Port 25/10 Gb NIC&ROCE 11 AIX, IBM i, and Linux 
SR/Cu Adapter 

EC2U 58FB PCle3 2-Port 25/10 Gb NIC&ROCE SR/Cu 18 AIX, IBM i, and Linux 
Adapter 

EC67 2CF3 PCle4 LP 2-port 100 Gb ROCE EN LP AIX, IBM i, and Linux 
adapter 

ENOS 2CC3 PCle2 4-Port (10 Gb+1 GbE) SR+RJ45 AIX, IBM i, and Linux 
Adapter 


ENOT 2CC3 PCle2 LP 4-Port (10 Gb+1 GbE) SR+RJ45 AIX, IBM i, and Linux 
Adapter 

ENOU 2CC3 PCle2 4-port (10 Gb+1 GbE) Copper AIX, IBM i, and Linux 
SFP+RJ45 Adapter 

ENOV 2CC3 PCle2 LP 4-port (10 Gb+1 GbE) Copper AIX, IBM i, and Linux 
SFP+RJ45 Adapter 


}ENOW | | 2cca | | PCle2 2-port 10/1 GbE BaseT RJ45 Adapter | 2-port 10/1 GbE BaseT RJ45 | PCle2 2-port 10/1 GbE BaseT RJ45 Adapter | 0 fis AIX, IBM i, } AIX, IBMi, and Linux | Linux 


een | — PCle2 LP 2-port 10/1 GbE BaseT RJ45 Hof A ein IBM i, and Linux 
Adapter 
EN15 2CE3 | PCle3 4-port 10 GbE SR Adapter as a AIX, IBM i, and Linux 


Table 2-22 lists the LAN adapters that are available for the Power H924 (9223-42S) server. 


Table 2-22 Available LAN adapters for the Power H924 (9223-42S) server 


Feature | CCIN Description OS support 
code 
5899 576F PCle2 4-port 1 GbE Adapter 26 AIX, IBM i, and 
Linux 
EC2S 58FA PCle3 2-Port 10 Gb NIC&ROCE SR/Cu 26 AIX, IBM i, and 
Adapter Linux 
EC2U 58FB PCle3 2-Port 25/10 Gb NIC&ROCE SR/Cu 26 AIX, IBM i, and 
Adapter Linux 
3 AIX, IBM i, and 
Linux 
AIX, IBM i, and 
Linux 


EC66 PCle4 2-port 100 Gb ROCE EN adapter a ic 
ENOS 2CC3 PCle2 4-Port (10 Gb+1 GbE) SR+RJ45 25 
Adapter 
ENOU 2CC3 PCle2 4-port (10 Gb+1 GbE) Copper 25 AIX, IBM i, and 
SFP+RJ45 Adapter Linux 
Pe 


ENOW 2CC4 PCle2 2-port 10/1 GbE BaseT RJ45 Adapter 
EN15 2CE3 PCle3 4-port 10 GbE SR Adapter 


AIX, IBM i, and 
Linux 
AIX, IBM i, and 
Linux 
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2.5.3 Graphics accelerator adapters 


An adapter can be configured to operate in 8-bit or 24-bit color modes. The adapter supports 
analog and digital monitors. 


Table 2-23 lists the graphics accelerator adapter that is available for the Power H922 
(9223-22S) server. 


Table 2-23 Available graphics accelerator adapter for the Power H922 server 


Feature | CCIN Description OS support 
code 


5269 5269 PCle LP POWER GXT145 Graphics AIX and Linux 
Accelerator 


Table 2-24 lists the graphics accelerator adapter that is available for the Power H924 
(9223-42S) server. 


Table 2-24 Available graphics accelerator card for the Power H924 server 


Feature | CCIN Description OS support 

code 

5748 5269 POWER GXT145 PCI Express Graphics 7 AIX and Linux 
Accelerator 


2.5.4 SAS adapters 


Table 2-25 lists the SAS adapters that are available for the Power H922 (9223-22S) server. 


Table 2-25 Available SAS adapters for the Power H922 (9223-22S) server 


Feature | CCIN Description OS support 

code 

EJOJ 57B4 PCle3 RAID SAS Adapter Quad-port 6 Gb x8 AIX, IBM i, and 

Linux 

EJOM 57B4 PCle3 LP RAID SAS Adapter Quad-Port 6 7 AIX, IBM i, and 
Gb x8 Linux 

EJ10 57B4 PCle3 SAS Tape/DVD Adapter Quad-port 6 12 AIX, IBM i, and 
Gb x8 Linux 

EJ11 57B4 PCle3 LP SAS Tape/DVD Adapter Quad-port 7 AIX, IBM i, and 
6 Gb x8 Linux 

EJ14 57B1 PCle3 12 GB Cache RAID PLUS SAS AIX, IBM i, and 
Adapter Quad-port 6 Gb x8 Linux 


Table 2-26 lists the SAS adapters that are available for the Power H924 (9223-42S) server. 


Table 2-26 Available SAS adapters for the Power H924 (9223-42S) server 


Feature | CCIN Description OS support 
code 


EJOJ 57B4_ | PCle3 RAID SAS Adapter Quad-port 6 Gb x8 cn ce AIX, IBM i, and 
Linux 
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Feature | CCIN Description OS support 
code 


EJ10 57B4 PCle3 SAS Tape/DVD Adapter Quad-port 6 19 AIX, IBM i, and 
Gb x8 Linux 

EJ14 57B1 PCle3 12 GB Cache RAID PLUS SAS 19 AIX, IBM i, and 
Adapter Quad-port 6 Gb x8 Linux 


2.5.5 Fibre Channel adapters 


The servers support direct or SAN connections to devices that use Fibre Channel (FC) 
adapters. 


Note: If you are attaching a device or switch with an SC type fiber connector, an LC-SC 
50-Micron Fiber Converter Cable (#2456) or an LC-SC 62.5-Micron Fiber Converter Cable 
(#2459) is required. 


Table 2-27 lists the FC adapters that are available for the Power H922 (9223-22S) server. 
They all feature LC connectors. 


Table 2-27 Available FC adapters for the Power H922 (9223-22S) server 


ENOA 577F PCle3 16 Gb 12 AIX, IBM i, and 
2-port Fibre Linux 
Channel Adapter 


ENOB 577F PCle3 LP 16 Gb AIX, IBM i, and 
2-port Fibre Linux 
Channel Adapter 


EN1A 578F PCle3 32 Gb 
2-port Fibre 
Channel Adapter 

EN1B 578F PCle3 LP 32 Gb 
2-port Fibre 
Channel Adapter 


AIX, IBM i, and 
Linux 


AIX, IBM i, and 
Linux 


AIX, IBM i, and 
Linux 


EN1D 578E PCle3 LP 16 Gb 
4-port Fibre 
Channel Adapter 

EN2A 579D PCle3 2-Port 16 
Gb Fibre Channel 
Adapter 

EN2B 579D PCle3 LP 2-Port 
16 Gb Fibre 
Channel Adapter 
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8 AIX, IBM i, and 
Linux 


AIX, IBM i, and 
Linux 


ae 
EN1C 578E PCle3 16 Gb 12 AIX, IBM i, and 
4-port Fibre Linux 
Channel Adapter 
a a 


Table 2-28 lists the FC adapters that are available for the Power H924 (9223-42S) server. 
They all feature LC connectors. 


Table 2-28 Available Fibre Channel adapters for the Power H924 (9223-42S) server 


Feature CCIN Description Operating system 

code support 

ENOA 577F | PCle3 16 Gb 2-port Fibre Channel Adapter a a AIX, IBM i, and 
Linux 


EN1A 578F PCle3 32 Gb 2-port Fibre Channel Adapter 25 AIX, IBM i, and 
Linux 

EN1C 578E PCle3 16 Gb 4-port Fibre Channel Adapter 25 AIX, IBM i, and 
Linux 

EN2A 579D | PCle3 2-Port 16 Gb Fibre Channel Adapter ee ae AIX, IBM i, and 
Linux 


Note: The use of N_Port ID Virtualization (NPIV) through the Virtual I/O Server (VIOS) 
requires an NPIV-capable FC adapter, such as the #ENOA. 


2.5.6 InfiniBand host channel adapter 


The InfiniBand Architecture (IBA) is an industry-standard architecture for server I/O and 
inter-server communication. It was developed by the InfiniBand Trade Association (IBTA) to 
provide the levels of reliability, availability, performance, and scalability that are necessary for 
present and future server systems with levels better than can be achieved by using 
bus-oriented |/O structures. 


InfiniBand is an open set of interconnect standards and specifications. The main InfiniBand 
specification is published by the IBTA and is available at the IBTA website. 


InfiniBand is based on a switched fabric architecture of serial point-to-point links. These 
InfiniBand links can be connected to host channel adapters (HCAs), which are used primarily 
in servers, or target channel adapters (TCAs), which are used primarily in storage 
subsystems. 


The InfiniBand physical connection consists of multiple byte lanes. Each individual byte lane 
is a four-wire, 2.5, 5.0, or 10.0 Gbps bidirectional connection. Combinations of link width and 
byte lane speed allow for overall link speeds of 2.5 - 120 Gbps. The architecture defines a 
layered hardware protocol and a software layer to manage initialization and the 
communication between devices. Each link can support multiple transport services for 
reliability and multiple prioritized virtual communication channels. 


For more information about InfiniBand, see HPC Clusters Using InfiniBand on IBM Power 
Systems Servers, SG24-7767. 


A connection to supported InfiniBand switches is accomplished by using the QDR optical 
cables (#3290 and #3293). 
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Table 2-29 lists the InfiniBand adapters that are available for the Power H922 (9223-22S) 
server. 


Table 2-29 Available InfiniBand adapters for the Power H922 (9223-22S) server 


Feature | CCIN | Description OS support 
code 


EC62 2CF1 PCle4 LP 1-port 100 Gb EDR InfiniBand CAPI 3 AIX, IBM i, 
adapter and Linux 
EC64 2CF2 PCle4 LP 2-port 100 Gb EDR InfiniBand CAPI AIX, IBM i, 
adapter and Linux 


Table 2-30 lists the InfiniBand adapter that is available for the Power H924 (9223-42S) server. 


Table 2-30 Available IInfiniBand adapter for the Power H924 (9223-42S) server 


Feature | CCIN | Description OS support 

code 

EC63 | 2CF1 | PCle4 1-port 100 Gb EDR InfiniBand CAPI adapter a a AIX, IBM i, and 
Linux 


2.5.7 Cryptographic coprocessor 


Table 2-31 lists the cryptographic coprocessor card that is available for the Power H922 
(9223-22S) server. 


Table 2-31 Available cryptographic coprocessor for the Power H922 (9223-22S) server 


Feature | CCIN | Description OS support 

code 

EJ33 | 4767 | PCle3 Crypto Coprocessor BSC-Gen3 4767 de ol AIX, IBM i, and 
Linux 


Table 2-32 lists the cryptographic coprocessor cards that are available for the Power H924 
(9223-42S) server. 


Table 2-32 Available cryptographic coprocessors for the Power H924 (9223-42S) server 


Feature | CCIN | Description OS support 
code 


EJ32 4767 PCle3 Crypto Coprocessor no BSC 4767 10 AIX, IBM i, and 
Linux 

EJ33 4767 PCle3 Crypto Coprocessor BSC-Gen3 4767 eee AIX, IBMi, and 
Linux 
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2.5.8 Coherent Accelerator Processor Interface adapters 


Table 2-33 lists the CAPI-capable adapters that are available for the Power H922 (9223-22S) 
server. 


Table 2-33 Available CAPI-capable adapters for the Power H922 (9223-22S) server 


Feature | CCIN Description OS support 
code 


EC62 2CF1 PCle4 LP 1-port 100 Gb EDR InfiniBand CAPI 3 AIX, IBMi, and 
adapter Linux 

EC64 2CF2 PCle4 LP 2-port 100 Gb EDR InfiniBand CAPI AIX, IBMi, and 
adapter Linux 


Table 2-34 lists the CAPI-capable adapter that is available for the Power H924 (9223-42S) 
server. 


Table 2-34 Available CAPI-capable adapter for the Power H924 (9223-42S) server 


Feature | CCIN Description OS support 

code 

EC63 2CF1 PCle4 1-port 100 Gb EDR InfiniBand CAPI adapter 3 AIX, IBM i, 
and Linux 


2.5.9 USB adapters 


Table 2-35 lists the USB adapters that are available for the Power H922 (9223-22S) server. 


Table 2-35 Available USB adapters for the Power H922 (9223-22S) server 


Feature | CCIN Description OS support 

code 

EC45 58F9 PCle2 LP 4-Port USB 3.0 Adapter AIX, IBM i, and 
Linux 

Ec46 | 58F9 | PCle2 4-Port USB 3.0 Adapter we AIX, IBMi, and 
Linux 


Table 2-36 lists the USB adapter that is available for the Power H924 (9223-42S) server. 


Table 2-36 Available USB adapter for the Power H924 (9223-42S) server 


Feature | CCIN Description OS support 
code 


EC46 58F9 PCle2 4-Port USB 3.0 Adapter 24 AIX, IBM i, 
and Linux 
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2.6 Internal storage 


The internal storage on the Power H922 and Power H924 servers depends on the 
DASD/Media backplane that is used. The servers support various DASD/Media backplanes. 


Power H922 (9223-22S) servers support the following backplanes: 


#EJ1F Base Storage Backplane with 8 SFF-3 Bays, which are accompanied 
by optional split card (#EJ1H). 

#EJ1G Expanded function Storage Backplane 8 SFF-3 Bays/Single IOA with 
write cache. 

#EJ1V Storage Backplane with 2 front PCle Gen4 capable NVMe U.2 drive 
slots. 

#EJ1W Storage Backplane with 4 front PCle Gen4 capable NVMe U.2 drive 
slots. 


Note: If the #EJ1G Expanded function backplane is ordered, the number of available x8 
PCle slots in the system unit is reduced by one. 


Note: If the new backplanes #EJ1V or #EJ1W are ordered, a minimum of one #ES1E, 
#ES1G, #EC5V, #EC5X (NVMe U.2 drive) must be ordered: 


> Maximum of two #ES1E, #ES1G, #EC5V, #EC5X per one #EJ1V. 
> Maximum of four #ES1E, #£S1G, #EC5V, #EC5X per one #EJ1W. 


Mixing of #ES1E, #ES1G, #EC5V, #EC5xX is allowed on a backplane. 


Power H924 (9223-42S) servers support the following backplanes: 


#EJ1C Base Storage Backplane with 12 SFF-3 Bays with an optional split 
card (#EJ1E). 

#EJ1D Expanded function Storage Backplane 18 SFF-3 Bays/Dual IOA with 
Write Cache and optional external SAS port. 

#EJ1M Expanded function Storage Backplane 12 SFF-3 Bays/RDX Bays with 
write cache and optional external SAS port and optional external SAS 
port. 

#EJ1S Storage Backplane with 6 SFF-3 Bays and 2 front PCle Gen4 capable 
NVMe U.2 drive slots. 

#EJ1T Storage Backplane with 2 front PCle Gen4 capable NVMe U.2 drive 
slots. 

#EJ1U Storage Backplane with 4 front PCle Gen4 capable NVMe U.2 drive 
slots. 
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Note: If the #EJ1D or # EJ1M Expanded function backplane is ordered with the optional 
#EJOW Two SAS ports feature, the number of available x8 PCle slots in the system unit is 
reduced by one. 


If the new backplanes #EJ1S, #EJ1T or #EJ1U are ordered, a minimum of one #ES1E, 
#ES1G, #EC5V, #EC5X (NVMe U.2 drive) must be ordered. 


> Maximum of two #ES1E, #ES1G, #EC5V, #EC5X per one #EJ1S or #EJ1T. 
> Maximum of four #ES1E, #£S1G, #EC5V, #EC5X per one #EJ1U. 


Mixing of #ES1E, #ES1G, #EC5V, #EC5xX is allowed on a backplane. 


Clients with write-sensitive disk or hard disk drive (HDD) workloads should consider 
upgrading from the base storage backplane (#EJ1F or #EJ1C) to the expanded function 
storage backplanes (#EJ1G, #EJ1D, or #EJ1M) to gain the performance advantage of write 
cache. 


2.6.1 Power H922 Base Storage Backplane (#EJ1F) 


The backplane option provides SFF-3 SAS bays in the system unit. These 2.5-inch or SFF 
SAS bays can contain SAS drives (HDDs or SSDs) that are mounted on a Gen tray or 
carrier. Therefore, the drives are designated SFF-3. SFF-1 or SFF-2 drives do not fit in an 
SFF-3 bay. All SFF-3 bays support concurrent maintenance or hot-plug capability. 


This backplane option uses leading-edge, integrated SAS RAID controller technology that is 
designed and patented by IBM. A custom-designed PowerPC based ASIC chip is the basis of 
these SAS RAID controllers. It provides RAID 0, RAID 5, RAID 6, and RAID 10 functions with 
HDDs and SSDs. Internally, SAS ports are implemented and provide plentiful bandwidth. The 
integrated SAS controllers are placed in dedicated slots and do not reduce the number of 
available PCle slots. 


The Storage Backplane option (#EJ1F) provides eight SFF-3 bays and one SAS controller 
with zero write cache. 


Optionally, by adding the Split Backplane (#EJ1H), a second integrated SAS controller with no 
write cache is provided, and the eight SSF-3 bays are logically divided into two sets of four 
bays. Each SAS controller independently runs one of the four-bay sets of drives. 


This backplane option supports HDDs or SSDs or a mixture of HDDs and SSDs in the SFF-3 
bays. Mixing HDDs and SSDs applies even within a single set of four bays of the split 
backplane option. If you are mixing HDDs and SSDs, they must be in separate arrays (unless 
you use the Easy Tier function). 


This backplane option can offer different drive protection options: RAID 0, RAID 5, RAID 6, or 
RAID 10. RAID 5 requires a minimum of three drives of the same capacity. RAID 6 requires a 
minimum of four drives of the same capacity. RAID 10 requires a minimum of two drives. 
Hot-spare capability is supported by RAID 5, RAID 6, or RAID 10. 


RAID 5 and RAID 6 result in more drive write activity than mirroring or unprotected drives. 
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The following operating systems are supported: 


Red Hat Enterprise Linux 
SUSE Linux Enterprise Server 
AIX 

IBM i (requires VIOS) 


vvvy 


The internal connections to the physical disks are shown in Figure 2-16. 
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Figure 2-13 #EJ1F connections 


2.6.2 Power H922 Split Backplane option for Base Storage Backplane (#EJ1H) 


This #EJ1H feature modifies the #EJ1F base storage backplane cabling and adds a second, 
high-performance SAS controller. The 8 SFF-3 SAS bays are cabled to be split into two sets 
of four bays, each with one SAS controller. Both SAS controllers are in integrated slots and do 
not use a PCle slot. 


The high-performance SAS controllers each provides RAID-0, RAID-5, RAID-6, and RAID-10 
support. JBOD support for HDD is also supported. No write cache is available on either 
controller. 


Both 5xx and 4k byte sector HDD/SSD are supported. 5xx and 4k drives cannot be mixed in 
the same array. 


The following are operating systems are supported: 


Red Hat Enterprise Linux 
SUSE Linux Enterprise Server 
AIX 

IBM i (requires VIOS) 


vvvy 
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The internal connections to the physical disks are shown in Figure 2-14. 
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Figure 2-14. #EJ1H connections 


2.6.3 Power H922 Expanded Function Storage Backplane (#EJ1G) 


Storage backplane with an integrated SAS controller with effectively up to 7.2 GB write cache. 
High-performance controllers run SFF-3 SAS bays in the system unit. Write cache power 
protection hardware covers one PCle x8 slot. Write cache augments a controller's high 
performance for workloads with writes, especially for HDD. 


A total of 1.8 GB physical write cache is used with compression to provide up to 7.2 GB cache 
capacity. The write cache contents are protected against power loss with flash memory and 
super capacitors, which remove the need for battery maintenance. This adapter uses 
non-mirrored write cache in a single adapter configuration. 


The high-performance SAS controllers provide RAID 0, RAID 5, RAID 6, RAID 10, RAID 5T2, 
RAID 6T2, and RAID 10T2 support. Patented Active/Active configurations with at least two 
arrays is supported. 


Easy Tier function is supported so the controller can automatically move hot data to attached 
SSD and cold data to attached HDD for AIX, Linux, and VIOS environments. 


Small Form Factor (SFF) or 2.5-inch drives are mounted on a carrier or tray specific to the 


system unit (SFF-3). The backplane has eight SFF-3 bays; all SFF-3 bays support concurrent 
maintenance or hot-plug capability. 
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This backplane also provides for an optional SAS port on the rear of the system unit that 
supports the attachment of one EXP12SX (#ESLL) or EXP24SX (#ESLS) SAS Storage 
Expansion Drawer Enclosure. One SAS YO cable ordered separately with mini-SAS HD 
narrow connector is used for the optional EXP12SX or EXP24SX attachment. This YO cable 
must be 3m or shorter. If shorter, remember that the cable management arm needs 
approximately 1 meter. 


This backplane does not support split backplane. To configure a split backplane, use 
backplane features #EJ1F with #EJ1H. 


Both 5xx and 4k byte sector HDD and SSD are supported. The 5xx and 4k drives cannot be 
mixed in the same array. 


Note: This backplane option reduces the number of available x8 PCle slots in the system 
unit by one. 


The following operating systems are supported: 


> Red Hat Enterprise Linux 
> SUSE Linux Enterprise Server 
> AIX 


The internal connections to the physical disks are shown in Figure 2-15. 
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Figure 2-15 #EJ1G connections 
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2.6.4 Power H922 NVMe Storage Backplanes with front U.2 drive slots (#EJ1V 
and #EJ1W) 


Non-Volatile Memory Express (NVMe) is a high-speed, low-latency storage media. A 
traditional HDD or SDD drive is connected to a SAS port of a SAS adapter, which is in a Gen3 
PCle slot. The NVMe drive is directly attached to the Gen4 PCle bus. It helps to reduce the 
protocol overhead of the SAS interface. 


Feature codes #EJ1V and #EJ1W represent a storage backplane with one or two PCle Gen4 
NVMe U.2 pass-thru cards for U.2 drives in the system unit’s front drive in slots. Each NVMe 
U.2 pass-thru card support two NVMe U.2 drive. 


In Power H922 (9223-22S) server the U.2 drives are connected by way of the NVMe Storage 
Backplane to a NVMe U.2 pass-thru card, which is in an internal Gen4 PCle slot of the 
machine. 

Power H922 (9223-22S) has following NVMe Storage Backplane options: 

> #EJ1V: Storage Backplane Gen 4 with 2 front NVMe U.2 drive slots 


The U.2 drives can be installed into system unit’s front drive in slot location codes P2-C1 
and C2. 


> #EJ1W: Storage Backplane Gen 4 with 4 front NVMe U.2 drive slots 
The U.2 drives can be installed into system unit’s front drive in slot location codes P2-C1 - 
C4. 

The following operating systems are supported: 


Red Hat Enterprise Linux 
SUSE Linux Enterprise Server 
AIX 

IBM i (requires VIOS) 


vvvy 


Note: In case of #EJ1V feature order, a #EJ1Q MES only option is available to install a 
second NVMe U.2 pass-thru card to increase the system’s U.2 drive capacity to four 
(technically equivalent to #EJ1W). 


No other HDD/SDD drives are allowed on a system with any of the NVMe Drives in the 
case of #EJ1V or #EJ1W backplanes. 


2.6.5 Power H924 Base Storage Backplane (#EJ1C) 


This feature code is the base storage backplane with an integrated SAS controller for SAS 
bays in the system unit. SAS bays are 2.5-inch or small form factor (SFF) and use drives that 
are mounted on a carrier or tray that is specific to the system unit (SFF-3). 


The high-performance SAS controller provides RAID 0, RAID 5, RAID 6, or RAID 10 support 
for HDDs or SSDs. JBOD support for HDD is also supported. The controller has no write 
cache. 


For servers that support split backplane capability, add #EJ1E. For write cache performance, 
use #EJ1D or #EJ1M instead of this backplane. 


5xx and 4-KB sector HDDs and SSDs are supported. 5xx and 4-KB drives cannot be mixed in 
the same array. 
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Feature code #EJIC provides a storage backplane with one integrated SAS adapter with no 
cache, running 12 SFF-3 SAS bays in the system unit and one RDX bay in the system unit. 


The following operating systems are supported: 


Red Hat Enterprise Linux 
SUSE Linux Enterprise Server 
Ubuntu Server 

AIX 

IBM i 


vvvvy 


The internal connections to the physical disks are shown in Figure 2-16. 
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Figure 2-16 #EJ1C connections 


2.6.6 Power H924 Split Backplane option for Base Storage Backplane (#EJ1E) 


This feature modifies the base Storage backplane cabling and adds a second, 
high-performance SAS controller. The 12 SFF-3 SAS bays are cabled to be split into two sets 
of six bays, each with one SAS controller. Both SAS controllers are in integrated slots and do 
not use a PCle slot. 


The high-performance SAS controllers each provide RAID-0, RAID-5, RAID-6, and RAID-10 
support. JBOD support for HDD is also supported. Write cache is not available on either 
controller. 


5xx and 4k byte sector HDD am SSD are supported. 5xx and 4k drives cannot be mixed in the 
same array. 


The following operating systems are supported: 
> Red Hat Enterprise Linux 


90 IBM Power Systems H922 and H924 Technical Overview and Introduction 


> SUSE Linux Enterprise Server 
> AIX 
> IBMi 


The internal connections to the physical disks are shown in Figure 2-17. 
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Figure 2-17 #EJ1E connections 


2.6.7 Power H924 Expanded Function Storage Backplanes (#EJ1D and #EJ1M) 


Feature codes #EJ1D and #EJ1M are expanded function storage backplanes with dual 
integrated SAS controllers with write cache and an optional external SAS port. 
High-performance controllers run SFF-3 SAS bays in the system unit. Dual controllers (also 
called dual I/O adapters or paired controllers) and their write cache are placed in integrated 
slots and do not use PCle slots. 


The write cache augments the controller's high performance for workloads with writes, 
especially for HDDs. A 1.8 GB physical write cache is used with compression to provide up to 
7.2-GB cache capacity. The write cache contents are protected against power loss by using 
flash memory and super capacitors, which remove the need for battery maintenance. 


These high-performance SAS controllers provide RAID 0, RAID 5, RAID 6, RAID 10, RAID 
5T2, RAID 6T2, or RAID 10T2 support. Patented active/active configurations with at least two 
arrays are supported. 


The Easy Tier function is supported so that the dual controllers can automatically move hot 
data to attached SSDs and cold data to attached HDDs for AIX, Linux, and VIOS 
environments. 


SFF or 2.5-inch drives are mounted on a carrier or tray that is specific to the system unit 
(SFF-3). 


Chapter 2. Architecture and technical overview 91 


These backplanes do not support a split backplane. For a split backplane, use #EJ1C plus 
#EJ1E. 


Both 5xx and 4-KB sector HDDs and SSDs are supported. 5xx and 4-KB drives cannot be 
mixed in the same array. 


Both expanded function storage backplanes are supported by following operating systems: 


Red Hat Enterprise Linux 
SUSE Linux Enterprise Server 
Ubuntu Server 

AIX 

IBM i 


vvvvy 


18 SFF-3 Bays/Dual IOA with Write Cache/Opt Ext SAS port (#EJ1D) 


The #EJ1D feature code provides a storage backplane with a pair of integrated SAS adapters 
with write cache, with an optional external SAS port running up to: 


> Aset of 18 SFF-3 SAS bays in the system unit. 


> Two SAS ports (#EJOW) at the rear of the system unit to connect to a single 
EXP24S/EXP24SxX I/O drawer. #EJOW is an optional feature with #EJ1D. One x8 PCle 
slot is used by #EJOW. 


The internal connections to the physical disks are shown in Figure 2-18. 
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Figure 2-18 #EJ1D connections 


92 IBM Power Systems H922 and H924 Technical Overview and Introduction 


12 SFF-3 Bays/Dual IOA with Write Cache/RDX/Opt Ext SAS port 
(#EJ1M) 


The #EJ1M feature code provides a storage backplane with a pair of integrated SAS adapters 


with write cache, a Removable Disk EXchange (RDX) bay, and an external SAS port running 
up to: 


> Aset of 12 SFF-3 SAS bays in the system unit. 
> One RDX bay in the system unit. 


> Two SAS ports (#EJOW) at the rear of the system unit to connect to a single 


EXP24S/EXP24SxX I/O drawer. #EJOW is an optional feature with #EJ1M. One x8 PCle 
slot is used by #EJOW. 


The internal connections to the physical disks are shown in Figure 2-19. 
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Figure 2-19 #EJ1M connections 


2.6.8 Power H924 NVMe Storage Backplanes with front U.2 drive slots (#EJ1T 
and #EJ1U) 


Feature codes #EJ1T and #EJ1U represent a storage backplane with one or two PCle Gen4 
NVMe U.2 pass-thru cards for U.2 drives in the system unit’s front drive in slots. Each NVMe 
U.2 pass-thru card support two NVMe U.2 drive. 


In Power H924 (9223-42S) server, the U.2 drives are connected by way of the NVMe Storage 


Backplane to a NVMe U.2 pass-thru card, which is in an internal Gen4 PCle slot of the 
machine. 
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Power H924 (9223-42S) has following NVMe Storage Backplane options: 
> #EJ1T: Storage Backplane Gen 4 with 2 front NVMe U.2 drive slots 


The U.2 drives can be installed into system unit’s front drive in slot location codes P2-C3 
and C4. 


> #EJ1U: Storage Backplane Gen 4 with 4 front NVMe U.2 drive slots 
The U.2 drives can be installed into system unit’s front drive in slot location codes P2-C1 - 
C4. 

The following operating systems are supported: 


Red Hat Enterprise Linux 
SUSE Linux Enterprise Server 
AIX 

IBM i (requires VIOS) 


vvvy 


Note: In case of #EJ1T feature order, a #EJ1Q MES only option is available to install 
second NVMe U.2 pass-thru card to increase the system’s U.2 drive capacity to four 
(technically equivalent to #EJ1U). 


No other HDD/SDD drives are allowed on a system with any of the NVMe Drives in case of 
#EJ1T or #EJ1U backplanes. 


2.6.9 Power H924 SAS and NVMe Backplane with front drive slots (#EJ1S) 


The #EJ1S feature code is a combined storage backplane with an integrated SAS controller 
for 6 SAS drives in the system unit’s front drive in slot, and with a PCle Gen4 NVMe U.2 
pass-thru card for 2 U.2 drives in the system unit’s front drive in slot. 


The SAS bays are 2.5-inch or small form factor (SFF) and use drives that are mounted on a 
carrier or tray that is specific to the system unit (SFF-3). SAS drives can be installed into 
system unit’s front drive in slot location codes P2-D1 - D6. 


The NVMe U.2 drives can be installed into system unit’s front drive in slot location codes 
P2-C3 and C4. 


The following operating systems are supported: 


Red Hat Enterprise Linux 
SUSE Linux Enterprise Server 
AIX 

IBM i (requires VIOS) 


vvvy 


2.6.10 RAID support 


Many protection options are available for HDDs and SSDs in the Power H922 and 

Power H924 servers, whether they are contained in the SAS SFF bays in the system unit or 
are drives in disk-only I/O drawers. Although protecting drives is always preferred, AIX and 
Linux users can choose to leave a few or all drives unprotected at their own risk. IBM 
supports these configurations. 
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Drive protection 
HDD and SSD protection can be provided by AIX, IBM i, and Linux, or by the HDD and SSD 
hardware controllers. 


Apart from NVMe backplane options, all of the storage backplanes with SAS bays offer RAID. 
The default storage backplanes contain one SAS HDD and SSD controller and support JBOD 
and RAID 0, 5, 6, and 10 for AIX or Linux. A secondary non-redundant controller is added 
when you use the split backplane option, so each of the six disk bays has a separated disk 
controller. 


When you choose the optional #EJ1D or #EJ1M storage backplane, the controller is replaced 
by a pair of high-performance RAID controllers with dual integrated SAS controllers with 

1.8 GB of physical write cache. High-performance controllers run 18 SFF-3 SAS bays with 
1.8-inch 


SSD bays. Dual controllers (also called dual I/O adapters or paired controllers) and their write 
cache are placed in integrated slots and do not use PCle slots. Patented active/active 
configurations with at least two arrays are supported. 


The write cache, which is responsible for increasing write performance by caching data 
before it is written to the physical disks, can have its data compression capabilities activated, 
which provides up to 7.2 GB effective cache capacity. The write cache contents are protected 
against power loss by flash memory and super capacitors, which removes the need for 
battery maintenance. 


The high-performance SAS controllers provide RAID 0, RAID 5, RAID 6, and RAID 10 
support, and the Easy Tier variants (RAID 5T2, RAID 6T2, and RAID 10T2) if the server has 
HDDs and SSDs installed. 


The Easy Tier function is supported, so the dual controllers can automatically move hot data 
to an attached SSD and cold data to an attached HDD for AIX and Linux, and VIOS For more 
information about Easy Tier, see 2.6.11, “Easy Tier’ on page 96. 


AIX and Linux can use disk drives that are formatted with 512-byte blocks when they are 
mirrored by the operating system. These disk drives must be reformatted to 528-byte sectors 
when used in RAID arrays. Although a small percentage of the drive’s capacity is lost, extra 
data protection, such as error-correcting code (ECC) and bad block detection, is gained in 
this reformatting. For example, a 300 GB disk drive provides approximately 283 GB when 
reformatted. IBM i always uses drives that are formatted to 528 bytes. SSDs are always 
formatted with 528-byte sectors. 


Supported RAID functions 


The base hardware supports RAID 0, 5, 6, and 10. When more features are configured, the 
server supports hardware RAID 0, 5, 6, 10, 5T2, 6T2, and 10T2: 


> RAID 0 provides striping for performance, but does not offer any fault tolerance. 


The failure of a single drive results in the loss of all data on the array. This version of RAID 
increases I/O bandwidth by simultaneously accessing multiple data paths. 


> RAID 5 uses block-level data striping with distributed parity. 


RAID 5 stripes both data and parity information across three or more drives. Fault 
tolerance is maintained by ensuring that the parity information for any block of data is 
placed on a drive that is separate from the ones that are used to store the data itself. This 
version of RAID provides data resiliency if a single drive fails in a RAID 5 array. 
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> RAID 6 uses block-level data striping with dual distributed parity. 


RAID 6 is the same as RAID 5, except that it uses a second level of independently 
calculated and distributed parity information for more fault tolerance. A RAID 6 
configuration requires N+2 drives to accommodate the extra parity data, which makes it 
less cost-effective than RAID 5 for equivalent storage capacity. This version of RAID 
provides data resiliency if one or two drives fail in a RAID 6 array. When you work with 
large capacity disks, RAID 6 enables you to sustain data parity during the rebuild process. 


> RAID 10 is a striped set of mirrored arrays. 


This set is a combination of RAID 0 and RAID 1. A RAID 0 stripe set of the data is created 
across a two-disk array for performance benefits. A duplicate of the first stripe set is then 
mirrored on another two-disk array for fault tolerance. This version of RAID provides data 
resiliency if a single drive fails, and it can provide resiliency for multiple drive failures. 


RAID 5T2, RAID 6T2, and RAID 10T2 are RAID levels with Easy Tier enabled. They require 
that both types of disks exist on the system under the same controller (HDDs and SSDs) and 
are configured under the same RAID type. 


2.6.11 Easy Tier 


96 


With a standard backplane (#EJ1F or #EJ1C), the server can handle HDDs and SSDs that 
are attached to its storage backplane if they are on separate arrays. 


The high-function backplane (#EJ1D) can handle both types of storage in the following ways: 


>» Separate arrays: SSDs and HDDs coexist on separate arrays, as can the Standard SAS 
adapter. 


> Easy Tier: SSDs and HDDs coexist under the same array. 


When the SSDs and HDDs are under the same array, the adapter can automatically move the 
most accessed data to faster storage (SSDs) and less accessed data to slower storage 
(HDDs). This situation is called Easy Tier. 


Coding or software intervention is not needed after the RAID is configured correctly. Statistics 
on block accesses are gathered every minute. After the adapter realizes that some portion of 
the data is frequently requested, it moves this data to faster devices. The data is moved in 
chunks of 1 MB or 2 MB, which are called bands. 


From the operating system perspective, only a regular array disk is used. From the SAS 
controller perspective, two arrays are used with parts of the data that is serviced by one tier of 
disks and parts by another tier of disks. 
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Figure 2-20 shows an Easy Tier array. 
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Figure 2-20 Easy Tier array 


The Easy Tier configuration is accomplished through a standard OS SAS adapter 
configuration utility. Figure 2-21 and Figure 2-22 on page 98 show two examples of tiered 
array creation for AIX. 


IBM SAS Disk Array Manager 


Move cursor to desired item and press Enter. 
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Move cursor to desired item and press Enter. 
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Figure 2-21 Array type selection pane on AIX RAID Manager 
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Nome Resource State Description Size 

sissasi FEFFFFFF Primary PCle3 12GB Cache RAID oe Rigas Quad-port 66b x8 

sissaso FEFFFFFF WA Linked Remote adapter SN 003250) 

hdisk1i FCOOOOFF Optimal RAID 5T2 Array (N/N) 773, 56GB RAID 5T2 
pdiskO O00400FF Active frray Member 139. 6G6B 
pdiskli GO0O401FF Active frray Member 139.6GB RAID 5 SSD 
pdisk2 000402FF Active frray Member 139.66B - 241% 177.8 GB 
pdisk3 O00403FF Active Array Member 139.668 RAID 5 HDD 
pdisk7 OOO407FF Active SSO Array Member 177. 8GB 341% 139.6GB 
pdisk6 GOO406FF Active SSO Array Member 177. 8GB = x : 
pdisks O00408FF Active SSO Array Member 177.868 

hdisk2 FCO1O0FF Optimal RAID 612 Array (N/N) 1e90GB <—————— RAID 6T2 
pdiski@ OO040AFF Active SSO Array Member 387.9GB 
pdiskil O@6040BFF Active SSO Array Member 387. 96B 
pdisk4 O00404FF Active Array Member 139. 66GB 
pdisk20 O00414FF Active SSO Array Member 387 .9GB RAID 6 SSD 
pdisk21 O0G6415FF Active SSO Array Member 387.9G6B 
pdisk9  ©00409FF Active SSO Array Member 177. 86B - 3+2x 387.9GB 
pdiskS O00405FF Active Array Member 139.6GB RAID 6 HDD 
pdiski2 OO0040CFF Active Array Member 139.6GB - 44+2x 139.6 GB 
pdiski3 O0040DFF Active Array Member 139.6G6B 
pdiski4 OO040EFF Active Array Member 139.6GB 
pdiskiS OO040FFF Active frray Member 139.6GB 

hdisk3 FCO200FF Optimal RAID 10T2 bella (0/0) 666.6GB RAID 10T2 
pdisk22 OO0416FF Active SSO Array Mem! 387.9GB 
pdisk23 O©00417FF Active SSD Array euber 387.9GB RAID 10 SSD 
pdisk16 ©00410FF Active Array Member 139.668 - 1+1x387.9GB 
pdiski7 O00411FF Active Array Member 139.6GB RAID 10 HDD 
pdiski8 600412FF Active Array Member 139.6GB - 2+2%x 139.6 GB 
pdiski9 OOG6413FF Active Array Member 139.6GB 


Figure 2-22 Tiered arrays (RAID 5T2, RAID 6T2, and RAID 10T2) example on AIX RAID Manager 


To support Easy Tier, ensure that the server is running at least the following minimum levels: 


VIOS 2.2.3.3 with interim fix 1V56366 or later 
AIX 7.1 TL3 SP3 or later 

Red Hat Enterprise Linux 6.5 or later 

SUSE Linux Enterprise Server 11 SP3 or later 
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IBM i supports Easy Tier only through VIOS. 


2.6.12 External SAS ports 


The Expanded function Storage Backplanes offer a connection to an external SAS port: 


> Power H922: #EJ1G 
> Power H924: #EJ1D and #EJ1M 


Power H924 servers that use the high-performance RAID feature support two external SAS 
ports. The external SAS ports are used for expansion to an external SAS drawer. 


More drawers and the IBM System Storage 7226 Tape and DVD Enclosure Express (Model 
1U3) can be attached by installing more SAS adapters. 


Note: Only one SAS drawer is supported from the external SAS port of Expanded function 
Storage Backplanes. More SAS drawers can be supported through SAS adapters. 


2.6.13 Media drawers 


IBM multimedia drawers, such as the 7226-1U3 or 7214-1U2, or tape units, such as the 
TS2240, TS2340, TS3100, TS3200, and TS3310, can be connected by using external SAS 
ports. 
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2.6.14 External DVD drives 


A trend is to use good quality USB flash drives rather than DVD drives. Being mechanical, 
DVD drives are less reliable than solid-state technology. 


If you feel that a DVD drive is needed, IBM offers a stand-alone external USB unit (#EUAS5), 
which is shown in Figure 2-23. 


Figure 2-23 Stand-alone USB DVD drive with cable (fEUA5) 


Note: If you use an external or stand-alone USB drive, which does not have its own power 
supply, you should use a USB socket at the front of the system to ensure enough current is 
available. 


2.6.15 RDX removable disk drives 


These Power Systems servers support Removable Disk EXchange (RDX) removable disk 
drives, which are commonly used for quick backups. 


If #EJ1C or #EJ1M backplane is configured, an internal bay is available and can be populated 
by #EU00 docking station for RDX disk cartridges. It connects to an internal USB port in the 
system unit. 


An external or stand-alone RDX unit (#EUA4) also is available. 
Note: Feature #EU04 is not available in US, EMEA, or Japan. 


The available disk drives are listed in Table 2-37. 


Table 2-37 RDX disk drives 


#EU08 46C5377 320 GB Removable Disk Drive (supported only) 
#1107 46C5379 500 GB Removable Disk Drive 


#EU01 46C2335 1 TB Removable Disk Drive 
#EU15 4602831 1.5 TB Removable Disk Drive (supported only) 
#EU2T 46C2975 2 TB Removable Disk Drive 
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2.7 External IO subsystems 


This section describes the PCle Gen3 I/O expansion drawer that can be attached to the 
Power H922 and Power H924 servers. 


2.7.1 Peripheral Component Interconnect Express Gen3 I/O expansion drawer 


The PCle Gen3 I/O expansion (EMX0O) drawer is a 4U high, PCI Gen3-based, and 
rack-mountable I/O drawer. It offers two PCle fan-out modules (#EMXF or #EMXG). The PCle 
fan-out module provides six PCle Gen3 full-high full-length slots (two x16 and four x8). The 
PCle slots are hot-pluggable. 


The PCle fan-out module has two CXP ports, which are connected to two CXP ports ona 
PCle Optical Cable Adapter (#EJ05, #EJ07, or #EJ08, depending on the server that is 
selected). A pair of active optical CXP cables (AOCs) or a pair of CXP copper cables are used 
for this connection. 


Concurrent repairing and adding or removing PCle adapters is done by using Hardware 
Management Console (HMC) guided menus or by operating system support utilities. 


A blind swap cassette (BSC) is used to house the full-high adapters that are installed in these 
slots. The BSC is the same BSC that is used with the previous generation server's 
#5802/5803/5877/5873 12X attached I/O drawers. 


Figure 2-24 shows the back view of the PCle Gen3 I/O expansion drawer. 


PCle Optical Interface to CEC PCle Optical Interface to CEC 


Fan-out Module Fan-out Module 
6 PCle G3 Slots 6 PCle G3 Slots 
4x8 &2x16 4x8 &2x16 Dual Power Cords 


Figure 2-24 Rear view of the PCle Gen3 I/O expansion drawer 
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2.7.2 PCle Gen3 I/O expansion drawer optical cabling 


I/O drawers are connected to the adapters in the system node through the following data 
transfer cables: 


> 3M Optical Cable Pair for PCle3 Expansion Drawer (#ECC7) 
> 10M Optical Cable Pair for PCle3 Expansion Drawer (#ECC8) 
> 3M Copper CXP Cable Pair for PCle3 Expansion Drawer (#ECCS) 


Cable notes: Use the 3.0 m cables for intra-rack installations. Use the 10.0 m cables for 
inter-rack installations. 


You cannot mix copper and optical cables on the same PCle Gen3 I/O drawer. Both fan-out 
modules use copper cables or both use optical cables. 


A minimum of one PCle3 Optical Cable Adapter for PCle3 Expansion Drawer is required to 
connect to the PCle3 6-slot fan-out module in the I/O expansion drawer. The fan-out module 
has two CXP ports. The top CXP port of the fan-out module is cabled to the top CXP port of 
the PCle3 Optical Cable Adapter. The bottom CXP port of the fan-out module is cabled to the 
bottom CXP port of the same PCle3 Optical Cable Adapter. 


To set up the cabling correctly, complete the following steps: 


1. Connect an optical cable or copper CXP cable to connector T1 on the PCle3 optical cable 
adapter in your server. 


2. Connect the other end of the optical cable or copper CXP cable to connector T1 on one of 
the PCle3 6-slot fan-out modules in your expansion drawer. 


3. Connect another cable to connector T2 on the PCle3 optical cable adapter in your server. 


4. Connect the other end of the cable to connector T2 on the PCle3 6-slot fan-out module in 
your expansion drawer. 


5. Repeat steps 1 - 4 for the other PCle3 6-slot fan-out module in the expansion drawer, if 
required. 


Drawer connections: Each fan-out module in a PCle3 Expansion Drawer can be 
connected only to a single PCle3 Optical Cable Adapter for PCle3 Expansion Drawer. 
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Figure 2-25 shows the connector locations for the PCle Gen3 I/O Expansion Drawer. 


P1-C1 
P1-C2 
P1-C3 
P1-C4 
P1-C5 
P1-C6 
P2-C2 
P2-C3 
P2-C4 
P2-C5 
P2-C6 
P8ECS902-1 


P2-C1 


Figure 2-25 Connector locations for the PCle Gen3 I/O expansion drawer 


Figure 2-26 shows the typical optical cable connections. 


Server 1/O Expansion Drawer 


Optical cables 


Optical cables : 


Figure 2-26 Typical optical cable connections 
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General rules for the PCI Gen3 I/O expansion drawer configuration 

The PCle3 Optical Cable Adapter for PCle3 Expansion Drawer (#EJ05) is supported in slots 
P1-C4 and P1-C9 for the Power H922 server. This feature code is a double-wide adapter that 
requires two adjacent slots. If #EJO5 is installed in this slot, the external SAS port is not 
allowed in the system. 


The PCle3 cable adapter for the PCle3 EMX0 expansion drawer (#EJ08) is supported in 
P1-C9 for the Power H924 single processor servers. It is supported in P1-C9, P1-C3, and 
P1-C4 in the Power H924 double processor servers. 


Table 2-38 lists PCle adapter slot priorities and maximum adapters that are supported in the 
Power H922, and Power H924 servers. 


Table 2-38 PCle adapter slot priorities and maximum adapters that are supported 


Feature code Slot priorities Maximum number of 
adapters supported 
Power H922 #EJ05 1 
(1 processor) 


Power H922 #EJ05 9/10, 3/4 2 
(2 processor) 
Power H924 #EJ08 1 
(1 processor) 
Power H924 #EJ08 9, 3, and 4 3 
(2 processor) 


2.7.3 PCle Gen3 I/O expansion drawer system power control network cabling 


No system power control network (SPCN) is used to control and monitor the status of power 
and cooling within the I/O drawer. SPCN capabilities are integrated into the optical cables. 


2.8 External disk subsystems 


This section describes the following external disk subsystems that can be attached to the 
Power H922, and Power H924 servers: 


> EXP12SX SAS Storage Enclosure (#ESLL) and EXP24SX SAS Storage Enclosure 
(#ESLS) 


> IBM Storage 


2.8.1 EXP12SX SAS Storage Enclosure and EXP24SX SAS Storage Enclosure 


The EXP24SX SAS Storage Enclosure is a storage expansion enclosure with 24 2.5-inch 
SFF SAS bays. It supports up to 24 hot-swap HDDs or SSDs in only 2 EIA of space in a 
19-inch rack. 


The EXP24SX SAS Storage Enclosure SFF bays use SFF Gen2 (SFF-2) carriers or trays that 
are identical to the carrier or trays in the previous EXP24S drawer. With AIX and Linux, or 
VIOS, the EXP24SX SAS Storage Enclosure can be ordered with four sets of six bays (mode 
4), two sets of 12 bays (mode 2), or one set of 24 bays (mode 1). With IBM i, one set of 24 
bays (mode 1) is supported. 
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HDDs and SSDs cannot be mixed in the same mode 1 drawer. HDDs and SSDs can be mixed 
in a mode 2 or mode 4 drawer, but they cannot be mixed within a logical split of the drawer. 
For example, in a mode 2 drawer with two sets of 12 bays, one set can hold SSDs and one 
set can hold HDDs, but you cannot mix SSDs and HDDs in the same set of 12 bays. 


The EXP12SX SAS Storage Enclosure is a storage expansion enclosure with 12 3.5-inch 
large form factor (LFF) SAS bays. It supports up to 12 hot-swap HDDs in only 2 EIA of space 
in a 19-inch rack. The EXP12SX SAS Storage Enclosure SFF bays use LFF Gen1 (LFF-1) 
carriers or trays. Only 4-KB sector (4096 or 4224) sector drives are supported. With AIX and 
Linux, and VIOS, the EXP12SX SAS Storage Enclosure can be ordered with four sets of 
three bays (mode 4), two sets of six bays (mode 2), or one set of 12 bays (mode 1). 


Four mini-SAS HD ports on the EXP12SX SAS Storage Enclosure and EXP24SX SAS 
Storage Enclosure or are attached to PCle Gen3 SAS adapters or attached to an integrated 
SAS controller in the Power H922, or Power H924 servers. 


The following PCle3 SAS adapters support the EXP12SX SAS Storage Enclosure and 
EXP24SX SAS Storage Enclosure: 


> PCle3 RAID SAS Adapter Quad-port 6 Gb x8 (#EJOJ, #EJOM, #EL3B, or #EL59) 
> PCle3 12 GB Cache RAID Plus SAS Adapter Quad-port 6 Gb x8 (#EJ14) 


Earlier generation PCle2 or PCle1 SAS adapters are not supported by the EXP24SX SAS 
Storage Enclosure. 


The attachment between the EXP12SX SAS Storage Enclosure or EXP24SX SAS Storage 
Enclosure and the PCle3 SAS adapters or integrated SAS controllers is through SAS YO12 
or X12 cables. All ends of the YO12 and X12 cables have mini-SAS HD narrow connectors. 
The following cable options are available: 


X12 cable: 3-meter copper (#ECDJ) 

YO12 cables: 1.5-meter copper (#ECDT) or 3-meter copper (HECDU) 
3M 100 GbE Optical Cable QSFP28 (AOC) (#EB5R) 

5M 100 GbE Optical Cable QSFP28 (AOC) (#EB5S) 

10M 100 GbE Optical Cable QSFP28 (AOC) (#EB5T) 

15M 100 GbE Optical Cable QSFP28 (AOC) (#EB5U) 

20M 100 GbE Optical Cable QSFP28 (AOC) (#EB5V) 

30M 100 GbE Optical Cable QSFP28 (AOC) (#EB5W) 

50M 100 GbE Optical Cable QSFP28 (AOC) (#EB5xX) 

100M 100 GbE Optical Cable QSFP28 (AOC) (#EB5Y) 
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Six SAS connectors are at the rear of the EXP12SX SAS Storage Enclosure and EXP24SX 
SAS Storage Enclosure to which SAS adapters or controllers are attached. They are labeled 
T1, T2, and T3; two T1, two T2, and two T3 connectors. 


Consider the following points: 


> In mode 1, two or four of the six ports are used. Two T2 ports are used for a single SAS 
adapter, and two T2 and two TS ports are used with a paired set of two adapters or a dual 
adapters configuration. 


> In mode 2 or mode 4, four ports are used, two T2 ports and two T3 ports to access all SAS 
bays. 


Figure 2-27 shows connector locations for the EXP12SX SAS Storage Enclosure and 
EXP24SX Storage Enclosure. 


P1-C2-T3 
P1-C2-T2 
P1-C2-T1 


P1-C1-T3 
P1-C1-T2 ——— 
P1-C1-T1 


Figure 2-27 Connector locations for the EXP12SX SAS Storage Enclosure and EXP24SX Storage 
Enclosure 


For more information about SAS cabling, see IBM Knowledge Center. 

The EXP12SX SAS Expansion Drawer and EXP24SX Expansion Drawer include the 
following high-reliability design points: 

> SAS bays that support hot-swap. 

> Redundant and hot-plug power and fan assemblies. 

> Dual power cords. 

> Redundant and hot-plug IBM Enclosure Services Manager (ESM). 

> Redundant data paths to all drives. 


> LED indicators on drives, bays, ESM, and power supplies that support problem 
identification. 


> Through the SAS adapters or controllers, drives that can be protected with RAID and 
mirroring and hot-spare capability. 
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IBM Storage Systems products and offerings provide compelling storage solutions with 
superior value for all levels of business, from entry-level to high-end storage systems. For 
more information about the various offerings, see Data Storage Solutions. 


The following sections describe some of these offerings. 


IBM Flash Storage 


The next generation of IBM Flash Storage delivers the performance and efficiency that you 
need to succeed with a new pay-as-you-go option to reduce your costs and 
scale-on-demand. 


For more information, see Flash Storage and All Flash Arrays. 


IBM DS8880 Hybrid Storage 

IBM DS8880 Hybrid Storage is a family of storage systems that includes the IBM DS8886 
storage system for high-performance functions in a dense, expandable package. The IBM 
DS8884 storage system provides advanced functions for consolidated systems or multiple 
platforms in a space-saving design. IBM DS8880 storage systems combine resiliency and 
intelligent flash performance to deliver microsecond application response times and more 
than six-nines availability. 


For more information, see IBM DS8880 hybrid storage - Overview. 


IBM FlashSystem 7200 


IBM FlashSystem® 7200 is an enterprise-class storage solution that offers the advantages of 
IBM Spectrum Virtualize software. It can help you lower capital and operational storage costs 
with heterogeneous data services while optimizing performance with flash storage. 

IBM FlashSystem 7200 enables you to take advantage of hybrid cloud technology without 
replacing your current storage. 


For more information, see IBM FlashSystem 7200 - Overview. 


IBM FlashSystem 5000 


IBM FlashSystem 5000 is a flexible storage solution that offers extraordinary scalability from 
the smallest to the largest system without disruption. Built with IBM Spectrum Virtualize 
software, it can help you lower capital and operational storage costs with heterogeneous data 
services. IBM FlashSystem 5000 is an easily customizable and upgradeable solution for 
better investment protection, improved performance, and enhanced efficiency. 


For more information, see IBM FlashSystem 5000 - Overview. 
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2.9 Operating system support 


Linux is required as the primary operating system with a minimum of 75% of total cores 
activated. 


IBM AIX and IBM i can be chosen as secondary operating systems, with a maximum of 25% 
of total cores activated across both. 


Note: If the AlX and IBM i total core activation requirements exceed 25%, the Power S924 
and S922 models are the recommended servers. 


To support SAP production implementations, use one of the following Linux operating system 
offerings: 


> SUSE Linux Enterprise Server for SAP with SUSE Linux Enterprise Server 15 Service 
Pack 1, or later 


> SUSE Linux Enterprise Server for SAP with SUSE Linux Enterprise Server 12 Service 
Pack 5, or a later 


> Red Hat Enterprise Linux for SAP with Red Hat Enterprise Linux 8 for Power LE, version 
8.1, or a later 


To support non-SAP production implementations, use one of the following Linux operating 
system offerings: 


> SUSE Linux Enterprise Server 15 Service Pack 1, or a later 
>» SUSE Linux Enterprise Server 12 Service Pack 5, or a later 
> Red Hat Enterprise Linux 8 for Power LE, version 8.1, or later 


If the AIX operating system LPAR is installed with any I/O configuration, the following levels 
are supported: 


> AIX Version 7.2 with the 7200-05 Technology Level, or later 


> AIX Version 7.2 with the 7200-04 Technology Level and Service Pack 7200-04-02-2028, or 
later 


> AIX Version 7.1 with the 7100-05 Technology Level and Service Pack 7100-05-06-2028, or 
later 


> AIX Version 7.2 with the 7200-03 Technology Level and Service Pack 7 (Planned 
Availability February 19, 2021 


To support the AIX operating system by using virtual I/O through the use of PowerVM, use 
one of the following operating system offerings: 


AIX Version 7.2 with the 7200-05 Technology Level, or later 

AIX Version 7.2 with the 7200-04 Technology Level, or later 

AIX Version 7.2 with the 7200-03 Technology Level, or later 

AIX Version 7.1 with the 7100-05 Technology Level and Service Pack 7100-05-02-1832, or 
later 
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lf Virtual I/O Server is installed, the following level is supported: 


> VIOS 3.1.2.1, or later 
> VIOS 3.1.1.25, or later 
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2.9.1 IBM i 


If IBM i is installed, the following IBM i operating system levels are supported: 


> IBMi 7.4 TR3, or later 
> IBMi 7.3 TR9, or later 
> IBMi 7.2 with 7.2 Licensed Machine Code - RS 720-Q, or later 


For more information about the software that is available on Power Systems, see the 
IBM Power Systems Software web page. 


IBM i is supported on the Power H922 and Power H924 servers with the following minimum 
required levels: 


> IBMi7.4TR3 
> IBMi7.3 TRI 
> IBMi 7.2 with 7.2 Licensed Machine Code - RS 720-Q, or later 


Note: The Power H922 server requires the VIOS to run IBM i. Up to 4-core per IBM i 
partition are supported. Multiple IBM i partitions can be created and run concurrently. 


IBM i does not support the 4-core processor feature EP76. 


Table 2-39 lists software tiers for the specific servers. 


Table 2-39 Server type and the software tier associated 


a as 


IBM periodically releases maintenance packages (service packs or technology levels) for the 
IBM i operating system. For more information about these packages, downloading, and 
obtaining the installation media, see IBM Fix Central. 


For compatibility information about hardware features and the corresponding AIX and IBM i 
Technology Levels, see IBM Prerequisites. 


Service and productivity tools 

Service and productivity tools are available ina YUM repository that you can use to download 
and then install all of the recommended packages for your Red Hat, SUSE Linux, or Fedora 
distribution. The packages are available from Service and productivity tools for Linux on 
Power servers. 


To learn about developing on the IBM Power Architecture®, find packages, get access to 
cloud resources, and discover tools and technologies, see Linux on IBM Power Systems 
Developer Portal. 


The IBM Advance Toolchain for Linux on Power is a set of open source compilers, runtime 
libraries, and development tools that users can use to take leading-edge advantage of 

IBM POWER hardware features on Linux. For more information, see Advanced Toolchain for 
Linux on Power. 
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For more information about SUSE Linux Enterprise Server, see SUSE Linux Enterprise 
Server. 


For more information about Red Hat Enterprise Linux, see Red Hat Enterprise Linux. 


2.9.2 Virtual I/O Server 


The Power H922 and Power H924 servers require the following minimum level of VIOS: 


> VIOS 3.1.2.10, or later 
> VIOS 3.1.1.25, or later 


IBM regularly updates the VIOS code. For more information about the latest updates, see 
Fix Central. 


2.10 Power Systems reliability, availability, and serviceability 
capabilities 


This section provides information about Power Systems reliability, availability, and 
serviceability (RAS) design and features. 


RAS includes the following elements: 


Reliability Indicates how infrequently a defect or fault in a server occurs. 

Availability Indicates how infrequently the functioning of a system or application is 
impacted by a fault or defect. 

Serviceability Indicates how well faults and their effects are communicated to system 
managers and how efficiently and nondisruptively the faults are 
repaired. 


Table 2-40 lists the Power Systems RAS capabilities by operating system. The HMC is an 
optional feature on scale-out Power Systems servers. 


Table 2-40 Selected RAS features by operating system 


[rasientre A 
First Failure Data Capture (FFDC) for fault detection/error isolation 


1/0 subsystem 
PCle bus enhanced error detection 


PCle bus enhanced error recovery 


x Xx 


Memory availability 


Fault detection and isolation 
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Storage Protection Keys X Not used by Not used by 
OS OS 


Error log analysis 


Serviceability 


Boot-time progress indicators 


EED collection 
SP/OS “Call Home” on non-HMC configurations 


IO adapter/device stand-alone diagnostic tests with PowerVM 
SP mutual surveillance with IBM POWER Hypervisor 


Service Agent Call Home Application 
Service Indicator LED support 
System dump for memory, POWER Hypervisor, and SP 


IBM Knowledge Center/IBM Systems Support Site service 
publications 


System Service/Support Education 


Dual disk controllers/Split backplane 


Dynamic firmware update with HMC 


OS error reporting to HMC Service Focal Point (SFP) application 


RMC secure error transmission subsystem 
Healthcheck scheduled operations with HMC 


Operator panel (real or virtual [HMC] 

Concurrent Op Panel Display Maintenance 

High availability (HA) clustering support 

Repair and Verify Guided Maintenance with HMC 

PowerVM Live Partition/Live Application Mobility With PowerVM 


Enterprise Edition 


Early Power Off Warning (EPOW) 
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2.11 Manageability 


Several functions and tools help with manageability so that you can efficiently and effectively 
manage your system. 


2.11.1 Service user interfaces 


The service user interface enables support personnel or the client to communicate with the 
service support applications in a server by using a console, interface, or terminal. Delivering a 
clear, concise view of available service applications, the service interface enables the support 
team to manage system resources and service information in an efficient and effective way. 
Applications that are available through the service interface are carefully configured and 
placed to give service providers access to important service functions. 


Various service interfaces are used, depending on the state of the system and its operating 
environment. The following primary service interfaces are available: 


> Light Path, which provides indicator lights to help a service technical find a component in 
need of service. 


> Service processor. 

> Advanced System Management Interface (ASMI). 

> Operator panel. 

> An OS service menu, which obtains error codes directly from the hardware. 
> SFP on the HMC. 


Service processor 


The service processor is a controller that is running its own operating system. It is a 
component of the service interface card. 


The service processor operating system has specific programs and device drivers for the 
service processor hardware. The host interface is a processor support interface that is 
connected to the POWER processor. The service processor is always working, regardless of 
the main system unit’s state. The system unit can be in the following states: 


> Standby (power off) 
> Operating, ready to start partitions 
> Operating with running LPARs 


The service processor is used to monitor and manage the system hardware resources and 
devices. The service processor checks the system for errors, ensures that the connection to 
the management console for manageability purposes is functioning, and accepts ASMI 
Secure Sockets Layer (SSL) network connections. The service processor can view and 
manage the machine-wide settings by using the ASMI, which enables complete system and 
partition management from the HMC. 


Analyzing a system that does not start: The FSP can analyze a system that does not 
start. Reference codes and detailed data are available in the ASMI and are transferred to 
the HMC. 
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The service processor uses two Ethernet ports that run at 1 Gbps speed. Consider the 
following points: 


> Both Ethernet ports are visible only to the service processor and can be used to attach the 
server to an HMC or to access the ASMI. The ASMI options can be accessed through an 
HTTP server that is integrated into the service processor operating environment. 


> Both Ethernet ports support only auto-negotiation. Customer-selectable media speed and 
duplex settings are not available. 


> Both Ethernet ports have a default IP address: 


— Service processor ethO (HMC1 port) is configured as 169.254.2.147. 
— Service processor eth1 (HMC2 port) is configured as 169.254.3.147. 
— DHCP by using the HMC for the HMC management networks is also possible. 


The following functions are available through the service processor: 


Call Home 

ASMI 

Error information (error code, part number, and location codes) menu 
View of guarded components 

Limited repair procedures 

Generate dump 

LED Management menu 

Remote view of ASMI menus 

Firmware update through a USB key 
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Advanced System Management Interface 
ASML is the interface to the service processor that enables you to manage the operation of 
the server, such as auto-power restart, and to view information about the server, such as the 


error log and Vital Product Data (VPD). Various repair procedures require connection to the 
ASMI. 


The ASMI is accessible through the management console. It is also accessible by using a 
web browser on a system that is connected directly to the service processor (in this case, 
either a standard Ethernet cable or a crossed cable) or through an Ethernet network. ASMI 
can also be accessed from an ASCII terminal, but it is available only while the system is in the 
platform powered-off mode. 
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Figure 2-28 shows a method of opening ASMI on a specific server by using the HMC GUI. 


hmc15 Resources = AllSystems + 


All Systems 


View and monitor the state, health, and capacity information of all the systems that are connected to the management console 


Select All 7 View System Partitions 
View System Properties 


View Performance Dashboard 


Performance Data Collection » 


P9-S922-amber 


Me eG 


Schedule Operations... 
Updates » 


Serviceable Events Manager. 


Reference Code Log 


() Operating 
Aa 13% Reset or Remove System Connection 
a 13% 
0% Operations Attention 
“fe 
=| 30% Power Off Identify At 
Power Management Test Atten 


Schedule Operations 


Launch Advanced System 
Management (ASM) 


Rebuild System 
Change System Password 


Templates Updates 


Figure 2-28 Starting the ASMI by using the HMC GUI 


You are prompted for confirmation about which FSP to use, and then, a login window opens, 
as shown in Figure 2-29. 


i) My https://hmc15.aixncc.uk.ibm.com/hmc/content?taskld=67&refresh=168 80% | wily 


Advanced Sy 


P9-S$924-red 


User ID Welcome 


Machine type-model: 9009-42A 

Serial number: 7804930 

Date: 2018-6-1 

Time: 12:53:45 UTC 

Service Processor: Primary (Location: U78D2.001.WZS00JM-P1-C1) 


Password 
Language 


The User Status table which displays user ids and the status of user ids are only visible to users 
logged in with at least admin privileges. 


Figure 2-29 ASMI login window 
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After you are logged in (the default credentials are admin/admin), you see the menu that is 
shown in Figure 2-30. 


Advanced System Management 


User ID: admin P9-$924-red FW910 
Update Access Key Exp Date (YYYY- 


4 Expand all menus Welcome 


= Power/Restart Control 
System Service Aids 
System Information 

+4] System Configuration 
= Network Services 
Performance Setup 
On Demand Utilities 
Login Profile 


Machine type-model: 9009-42A 

Serial number: 7804930 

Date: 2018-6-1 

Time: 12:54:44 UTC 

Service Processor: Primary (Location: U78D2.001.WZS00JM-P1-C1) 


Current hardware uptime: 22 Days 1 Hours 27 Minutes 51 Seconds 
Current firmware uptime: 22 Days 1 Hours 24 Minutes 21 Seconds 


Current users 
User ID| Location 
admin ||10.255.128.1 


User Status 
User ID | Status 
dev Disabled 


celogin |Enabled 


celogin1 |Disabled 


celogin? |Disabled 


Figure 2-30 An ASMI window 


Tip: If you click Expand all menus, as shown in the red ring in Figure 2-30, you can use 
the search function (Ctrl+F) in your browser to find quickly menu items. 


Use the ASMI to change the service processor IP addresses or to apply certain security 
policies and prevent access from unwanted IP addresses or ranges. 


You might be able to use the service processor’s default settings. In that case, accessing the 
ASMI is not necessary. To access ASMI, use one of the following methods: 
> Use a management console. 


If configured to do so, the management console connects directly to the ASMI for a 
selected system from this task. 


To connect to the ASMI from a management console, complete the following steps: 
a. Open Systems Management from the navigation pane. 
b. From the work window, select one of the managed systems. 


c. From the System Management tasks list, click Operations + Launch Advanced 
System Management (ASM). 
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Use a web browser. 


At the time of this writing, the supported web browsers are Microsoft Internet Explorer 
(Version 10.0.9200.16439), Mozilla Firefox ESR (Version 24), and Chrome (Version 30). 
Later versions of these browsers might work, but are not officially supported. The 
JavaScript language and cookies must be enabled, and TLS 1.2 might need to be 
enabled. 


The web interface is available during all phases of system operation, including the initial 
program load (IPL) and run time. However, several of the menu options in the web 
interface are unavailable during IPL or run time to prevent usage or ownership conflicts if 
the system resources are in use during that phase. The ASMI provides an SSL web 
connection to the service processor. To establish an SSL connection, open your browser 
and go to the following address: 


https://<ip_address of _service_processor> 


Note: To make the connection through Internet Explorer, click Tools Internet Options. 
Clear the Use TLS 1.0 checkbox, and click OK. 


Use an ASCII terminal. 


The ASMI on an ASCII terminal supports a subset of the functions that are provided by the 
web interface. The ASMI is available only when the system is in the platform powered-off 
mode. The ASMI on an ASCII console is not available during several phases of system 
operation, such as the IPL and run time. 


Command-line start of the ASMI 


On the HMC itself or when correctly configured on a remote system, it is possible to start 
the ASMI web interface from the HMC command line. Open a terminal window on the 
HMC or access the HMC with a terminal emulation and run the following command: 


asmmenu --ip <ip address> 


On the HMC itself, a browser window opens automatically with the ASMI window. When 
configured correctly, a browser window opens on a remote system when it is issued from 
there. 


Operator panel 
The service processor provides an interface to the operator panel, which is used to display 
system status and diagnostic information. 


The operator panel has two parts; one is always installed, and the second can be optional. 


The part that is always installed provides the following LEDs and sensors: 


> 


Power LED: 


— Color: Green 

Off: Enclosure is off (AC cord is not connected) 

— On Solid: Enclosure is powered on 

On Blink: Enclosure is in the standby-power state 


Enclosure Identify LED. Color: Blue 
Enclosure Identify LED: 


— Color: Blue 
— Off: Normal 
— On Solid: Identify State 
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> System Fault LED: 


— Color: Amber 
— Off: Normal 
— On Solid: Check Error Log 


> System Roll-up LED: 


— Color: Amber 
— Off: Normal 
— On Solid: Fault 


> Power Button 

> System Reset Switch 

> Two Thermal Sensors 

> One Pressure/Altitude Sensor 

The LCD operator panel is optional in some systems; however, at least one must be used in a 
server in a rack that contains any Power H922 or Power H924 server. The panel can be 
accessed by using the switches that are on the front panel. 

The operator panel includes the following features: 

> Two x16 character LCD displays 

> Increment, decrement, and “enter” buttons 

The following functions are available through the operator panel: 


Error information 

Generate dump 

View machine type, model, and serial number 
Limited set of repair functions 
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The System Management Services (SMS) error log is accessible through the SMS menus. 
This error log contains errors that are found by partition firmware when the system or partition 
is starting. 


The service processor’s error log can be accessed through the ASMI menus. 


You can also access the system diagnostics from a Network Installation Manager (NIM) 
server. 


IBM i and its associated machine code provide dedicated service tools (DSTs) as part of the 
IBM i licensed machine code (Licensed Internal Code) and System Service Tools (SSTs) as 
part of IBM i. These interfaces can be used to check various system information. For more 
information, see Power Systems: IBM i service functions. 


Service Focal Point on the Hardware Management Console 


Service strategies become more complicated in a partitioned environment. The Manage 
Serviceable Events task in the HMC can help streamline this process. 


Each LPAR reports errors that it detects and forwards the event to the SFP application that is 
running on the management console without determining whether other LPARs also detect 
and report the errors. For example, if one LPAR reports an error for a shared resource, such 
as a managed system power supply, other active LPARs might report the same error. 


By using the Manage Serviceable Events task in the HMC, you can avoid long lists of 


repetitive Call Home information by recognizing that they are repeated errors and 
consolidating them into one error. 
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In addition, you can use the Manage Serviceable Events task to start service functions on 
systems and LPARs, including the exchanging of parts, configuring connectivity, and 
managing memory dumps. 


2.11.2 Power Systems Firmware maintenance 


The IBM Power Systems Client-Managed Microcode is a methodology that enables you to 
manage and installation microcode updates on Power Systems servers and their associated 
I/O adapters. 


Firmware entitlement 


Since POWERS, the server firmware updates are restricted to entitled servers. A customer 
must be registered with IBM and entitled by a service contract. 


During the initial machine warranty period, the access key is installed in the machine by 
manufacturing. The key is valid for the regular warranty period plus some extra time. 


The Power Systems Firmware is relocated from the public repository to the access control 
repository. The I/O firmware remains on the public repository, but the server must be entitled 
for installation. 


When the 1slic command is run to display the firmware levels, a new value, 
update_access_key_exp_date, is added. The HMC GUI and the ASMI menu show the 
“Update access key expiration date”. 


When the system is no longer entitled, the firmware updates fail. The following new System 
Reference Code (SRC) packages are available: 


> E302FA06: Acquisition entitlement check failed 
> E302FA08: Installation entitlement check failed 


Any firmware release that was made available during the entitled timeframe can still be 
installed. For example, if the entitlement period ends on 31 December 2019 and a new 
firmware release is released before the end of that entitlement period, it still can be installed. 
If that firmware is downloaded after 31 December 2019 but it was made available before the 
end of the entitlement period, it still can be installed. Any newer release requires a new 
update access key. 


Note: The update access key expiration date requires a valid entitlement of the system to 
perform firmware updates. 


For more information about an update access key, see this IBM Power Systems web page. 


For more information, see the wIBM Entitled Software Support web page. 


Firmware updates 

System firmware is delivered as a release level or a service pack. Release levels support the 
general availability (GA) of new functions or features, and new machine types or models. 
Upgrading to a higher release level is disruptive to customer operations. 


IBM intends to provide no more than two new release levels per year. These release levels 
are supported by service packs. Service packs are intended to contain only firmware fixes 
and not introduce new functions. A service pack is an update to a release level. 


Chapter 2. Architecture and technical overview 117 


If the system is managed by an HMC, the HMC is used for the firmware updates. With the 
HMC, you can take advantage of the Concurrent Firmware Maintenance (CFM) when 
available. With CFM, IBM increases its clients’ opportunity to stay on a release level for longer 
periods. Clients that expect maximum stability can defer the upgrade until a compelling 
reason to upgrade exists, such as the following examples: 


> Arelease level is approaching its end of service date (that is, it was available for 
approximately a year and soon service will not be supported). 


> Move a system to a more standardized release level when multiple systems are in an 
environment with similar hardware. 


> Anew release has a new function that is needed in the environment. 


> Ascheduled maintenance action causes a platform restart, which provides an opportunity 
to also upgrade to a new firmware release. 


The updating and upgrading of system firmware depends on several factors, such as whether 
the system is stand-alone or managed by a management console, the firmware that is 
installed, and the operating systems that are running on the system. For more information 
about these scenarios and the associated installation instructions, see the firmware section of 
Fix Central. 


You might also want to review the best practice white papers that are found at the Service and 
support best practices for Power Systems web page. 


Firmware update steps 


The system firmware consists of service processor microcode, Open Firmware microcode, 
and SPCN microcode. 


The firmware and microcode can be downloaded and installed from an HMC, a running 
partition, or USB port number 1 on the back (if that system is not managed by an HMC). 


Power Systems Firmware has a permanent firmware boot side (A side) and a temporary 
firmware boot side (B side). New levels of firmware must be installed first on the temporary 
side to test the update’s compatibility with existing applications. When the new level of 
firmware is approved, it can be copied to the permanent side. 


The firmware update procedure depends on whether the system is HMC managed. 
Instructions are available at IBM Knowledge Center: 

1. Go to POWERS® systems’ IBM Knowledge Center website. 

2. Select the suitable Power Server-specific version documentation: 


— 9223-22H (IBM Power System H922) 
— 9223-42H (IBM Power System H924) 


3. Navigate to following section: 


— For HMC-managed system, select Troubleshooting, service, and support —> 
Hardware Management Console service — Hardware Management Console 
service (Machine Type 7063) — Getting fixes. 


— For non-HMC managed system, select Troubleshooting, service, and support —> 
Getting fixes. 


If an HMC is available to manage the server, the HMC interface can be used to view the levels 
of server firmware and power subsystem firmware that are installed and that are available to 
download and install. 
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Each server has the following levels of server firmware and power subsystem firmware: 
> Installed level 


This level of server firmware or power subsystem firmware is installed into memory after 
the managed system is powered off and then powered on. It is installed on the temporary 
side of system firmware. 


> Activated level 


This level of server firmware or power subsystem firmware is active and running in 
memory. 


> Accepted level 


This level is the backup level of the server or power subsystem firmware. You can return to 
this level of server or power subsystem firmware if you decide to remove the installed 
level. It is installed on the permanent side of system firmware. 


Figure 2-31 shows the different levels in the HMC. 


System Information 


Target name: P9-S924-red P9-S924-red 
Concurrent LIC update status: Enabled P9-S922-amber 
Current LIC repository location: None SERIA | 
EC Machine Type/ Update Access Installed Activated Accepted Unactivated | Platform Update 
Number ‘IC Type esd Key Expiration Level Level Level Peele we Control 
Serial Number fee Level Level 
01VL910 Managed System Primary 9009-42A*7804930 07/24/2021 FW910.00 (073) FW910.00 (073) FW910.00 (073) FW910.00 (073) Management Console 
| | 
__Close || Help” 


Figure 2-31 Firmware levels 


IBM provides the CFM function on selected Power Systems servers. This function supports 
applying nondisruptive system firmware service packs to the system concurrently (without 
requiring a restart operation to activate changes). For systems that are not managed by an 
HMC, the installation of system firmware is always disruptive. 


The concurrent levels of system firmware can, on occasion, contain fixes that are known as 
deferred. These deferred fixes can be installed concurrently but are not activated until the 
next IPL. Any deferred fixes are identified in the Firmware Update Descriptions table of the 
firmware document. For deferred fixes within a service pack, only the fixes in the service pack 
that cannot be concurrently activated are deferred. 


Table 2-41 lists the file-naming convention for system firmware. 


Table 2-41 Firmware naming convention 


PPNNSSS_FFF_DDD 
Package identifier 


en Last disruptive fix pack 


Using the example in Table 2-41 on page 119, System Firmware 01VL940_050_027, as 
displayed on the Firmware Download page URL, is described as Release Level 940, Service 
Pack level 050 and last disruptive service pack level as 027. 
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An installation is disruptive if the following statements are true: 
> The release levels (SSS) of the currently installed and the new firmware differ. 


> The service pack level (FFF) and the last disruptive service pack level (DDD) are equal in 
the new firmware. 


Otherwise, an installation is concurrent if the service pack level (FFF) of the new firmware is 
higher than the service pack level that is installed on the system and the conditions for 
disruptive installation are not met. 


2.11.3 Concurrent Firmware Maintenance improvements 


Since POWER6, firmware service packs (updates) are concurrently applied and take effect 
immediately. Occasionally, a service pack is included where most of the features can be 
concurrently applied. However, because changes to some server functions (for example, 
changing initialization values for chip controls) cannot occur during operation, a patch in this 
area required a system restart for activation. 


With the Power-On Reset Engine (PORE), the firmware can now dynamically power off 
processor components, change the registers, and reinitialize while the system is running 
without discernible impact to any applications that are running on a processor. With PORE, 
you can potentially make concurrent firmware changes in POWERS, which in earlier designs 
required a restart to take effect. 


Activating new firmware functions requires the installation of a firmware release level 
(upgrades). This process is disruptive to server operations and requires a scheduled outage 
and full server restart. 


2.11.4 IBM Electronic Services and IBM Electronic Service Agent 


IBM transformed its delivery of hardware and software support services to help you achieve 
higher system availability. Electronic Services is a web-enabled solution that offers an 
exclusive, no extra charge enhancement to the service and support that is available for IBM 
servers. These services provide the opportunity for greater system availability with faster 
problem resolution and preemptive monitoring. The Electronic Services solution consists of 
the following separate but complementary elements: 


> Electronic Services news page 
> Electronic Service Agent 


Electronic Services news page 

The Electronic Services news page is a single internet entry point that replaces the multiple 

entry points that traditionally are used to access IBM internet services and support. With the 
news page, you can gain easier access to IBM resources for assistance in resolving technical 
problems. 


Electronic Service Agent 


The Electronic Service Agent (ESA) is software that is on your server. It monitors events and 
transmits system inventory information to IBM on a periodic, client-defined timetable. The IBM 
ESA automatically reports hardware problems to IBM. 
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Early knowledge about potential problems enables IBM to deliver proactive service that can 
result in higher system availability and performance. In addition, information that is collected 
through the IBM ESA is made available to IBM Support Services Representatives (IBM 
SSRs) when they help answer your questions or diagnose problems. Installation and use of 
IBM ESA for problem reporting enables IBM to provide better support and service for your 
IBM server. 


The IBM Electronic Support portal enables you to perform the following tasks: 


View service information that is reported by IBM Electronic Service Agent 
Search all content by using advanced search capabilities 

Open and manage service requests 

Receive support content notifications by platform or individual product 
View Call Home problem events 
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You can access the IBM Electronic Support portal at the following web addresses: 


> IBM Support website: A portfolio of tools and resources to keep your systems, software, 
and applications running smoothly. 


> IBM Support Guide web page: The support portal to view contracts, inventory, heartbeat, 
and so on, of your systems. 


Note: To use some of the functions that are found on the IBM Electronic Support portal, 
such as viewing service information or Call Home events, you must provide an IBM ID. 


Electronic Services features the following benefits: 
> Increased uptime 


The IBM ESA tool enhances the warranty or maintenance agreement by providing faster 
hardware error reporting and uploading system information to IBM Support, which can 
mean less time that is wasted monitoring the symptoms, diagnosing the error, and 
manually calling IBM Support to open a problem record. 


Its 24x7 monitoring and reporting means no more dependence on human intervention or 
off-hours customer personnel when errors are encountered in the middle of the night. 


> Security 


The IBM ESA tool is secure in monitoring, reporting, and storing the data at IBM. The IBM 
ESA tool securely transmits through the internet (HTTPS or VPN) or modem. It can be 
configured to communicate securely through gateways to provide customers a single point 
of exit from their site. 


Communication is one way. Activating IBM ESA does not enable IBM to call into a 
customer's system. System inventory information is stored in a secure database, which is 
protected behind IBM firewalls. It is viewable by the customer and IBM only. The 
customer’s business applications or business data are never transmitted to IBM. 


> More accurate reporting 


Because system information and error logs are automatically uploaded to the IBM Support 
center with the service request, customers are not required to find and send system 
information, which decreases the risk of misreported or misdiagnosed errors. 


When inside IBM, problem error data is run through a data knowledge management 
system, and knowledge articles are appended to the problem record. 
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> Customized support 


By using the IBM ID that you enter during activation, you can view system and support 
information by selecting My Systems at IBM Support. 


My Systems provides valuable reports about installed hardware and software by using 
information that is collected from the systems by IBM ESA. Reports are available for any 
system that is associated with the customers IBM ID. Premium Search combines the 
function of search and the value of IBM ESA information, providing advanced search of 
the technical support knowledge base. By using Premium Search and the IBM ESA 
information that was collected from your system, you can see search results that apply 
specifically to your systems. 


For more information about how to use the power of IBM Electronic Services, contact your 


IBM SSR or see IBM Support. 


Service Event Manager 


The Service Event Manager (SEM) enables the user to decide which of the serviceable 
events are called home by the IBM ESA. It is possible to lock specific events. Some 
customers might not allow data to be transferred outside of their company. After the SEM is 


enabled, the analysis of the possible problems might take longer. 


To work with SEM, use the following commands: 


> To enable SEM, run the following command: 


chhmc -c sem -s enable 


> To disable SEM mode and specify what state in which to leave the Call Home feature, run 


the following commands: 


chhmc -c sem -s disable --callhome disable 
chhmc -c sem -s disable --callhome enable 


The easiest way to set up the IBM ESA is by using the wizard in the HMC GUI, which is 


shown in Figure 2-32. 


hmc15 Serviceability > Service Management v 


Service Management 


Serviceability 
[2 Create Serviceable Event 
[4 Manage Dumps 


C2 Transmit Service Information 


(4 Format Media 


Connectivity 

C Electronic Service Agent Setup Wizard 
C4 Authorize User 

( Enable Electronic Service Agent 

[4 Manage Outbound Connectivity 

[2 Manage Inbound Connectivity 

[4 Manage Customer Information 

(2 Manage Serviceable Event Notification 


[4 Manage Connection Monitoring 


View, repair and close serviceable events, report a serviceable event problem, and manage connect 


Create a serviceable event to report a 
Use the Electronic Service Agent to cc 
Schedule or transmit service informa’ 


Format a USB flash memory device 


Activate the Electronic Service Agent t| 
Authorize users with service IDs of th 
Enable the creation of call home requ 
Configure and manage the call home 
Provide authorization to the service p 
View and modify administrator, syster 
Configure a method to receive notific: 


Configure timers to detect outages a 


Figure 2-32 Accessing the Electronic Service Agent setup wizard 
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The wizard guides the user through the necessary steps, including entering information about 
the location of the system, contact information, and other details. The user can select which 
HMC or HMCs should be used, as shown in Figure 2-33. 


Welcome Administrator and System Account 


Manage Outbound Connectivity 


Call Home Server Consoles 


Local Console 


Conseciney Configuration Connectivity 


The following call home server consoles may handle call home requests submitted by this console. 


V Use discovered call home server consoles 


Action Add... 


IP Address or Host Name Type 


[ff 9.137.62.30 discovered 


10.255.128.1 local 


9.137.62.13 discovered 


Comment 


hme14 


hmce13 


hme15 


Figure 2-33 Managing which HMCs are used for Call Home 


If you select a server, you can easily open the SEM, as shown in Figure 2-34. 


hme13 Resources> AllSystems v¥ 


All Systems 


P9-S922-amber 


<M © oF a 


View Performance Dashboard.. 


nm Performance Data Collection » 


Schedule Operations... 


P9-$924-red| Updates 


Reference Code Log... 


Ally 59 x 


View and monitor the state, health, and capacity information of all the systems that are connected to the management console. 


Select All View System Partitions 


View System Properties 


© Operating ( Operating 

QcPu 14.cPU — % Reset or Remove System Connection.. 
CB 0 View All Actions 

SOS™ Avsiasie | I 47% 


Figure 2-34 Opening the Serviceable Events Manager for a server 
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You can browse the SEM menus to see the events for this server, as shown in Figure 2-35. 


hme13 Resources> AllSystems ¥ 


All Systems Ally 9 x 
View and monitor the state, health, and capacity information of all the systems that are connected to the management console. 
Select All Actions + | Total: 2 Selected: o 
ial 
= = 
> ad Serviceability 
| Tasks Log P9-S924-red 


Console Events Log 


Serviceable Events Manager j 


Events Manager for Call Home | Operating 

Service Management --—— 10% 
_ meee =) 22% 

0% 


395°° 216.5 GB 
- Available me] 43% 


Figure 2-35 Serviceable events for this server 
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SAP HANA and Cloud Private 


The Power H922 and Power H924 are flexible servers that can be used in general enterprise 
computing needs; however, they excel when deployed for SAP HANA and Cloud Private 
services. 


In this chapter, we discuss these two applications. 


This chapter includes the following topics: 


> 3.1, “SAP HANA on POWER’ on page 126 
> 3.2, “Cloud-ready H922 and H924 servers” on page 127 
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3.1 SAP HANA on POWER 
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SAP HANA customers are moving their workloads to IBM Power Systems to take advantage 
of a hardware platform that was designed specifically to support big data and analytics 
workloads, while also providing enterprise-level mission-critical 24x7 reliability. 


One key benefit of running SAP HANA on IBM Power Systems is its unmatched flexibility, 
including support for virtualization that is ready for use. Another benefit is the ability to deploy 
as a tailored data center integration, rather than an appliance. This ability gives customers the 
flexibility to deploy on a wide range of IBM Power Servers, combined with various storage 
options. 


With support of SAP HANA Version 2, the IBM POWER platform introduced full data 
compatibility with x86 platforms. This compatibility increases flexibility on the business data 
level. It allows easy data migration from x86 to IBM POWER by using simple copy or backup 
and restore tasks. Even mixed SAP HANA replication clusters can be built for database 
resiliency. 


In terms of performance, the HANA on POWER solution set the performance world record on 
the two-tier SAP sales and distribution standard application benchmark with 16 sockets. 


Note: The full SAP report for E980 (9080-M9S) sales and distribution benchmark 
performance is available for download. 


Running SAP HANA on Power includes the following other key benefits: 


> Faster provisioning with built-in virtualization for scaling capacity at the push of a button 
with granularity as little as 0.01 cores or 1 GB. 


> Best scalability options with up to 28 TB scale-up in a single HANA 2.0 database and up to 
16 SAP HANA instances in a single enterprise server. 


> Maximum uptime with highest availability non-mainframe Linux platform for over a decade 
and live partition mobility for zero impact planned maintenance. 


> Faster insights with 1.8x more memory bandwidth than compared to the x86 
infrastructure. 


> By using built-in fault tolerance, redundancy, and self-healing capabilities, customers can 
maximize SAP HANA availability with IBM Power Systems RAS technologies. The 
following memory RAS features improve SAP HANA availability: 


— Chipkill memory and advanced error checking and correcting (ECC) technology that 
with which customers can protect data stores from single memory chip failure 


— Extra chip per memory rank, which enables built-in memory rank sparing 


> SAP HANA on POWER can scale up or scale out. It also allows customers to have a mix 
of traditional SAP workloads and non-production workloads running in a shared pool, side 
by side with SAP HANA production workloads. 


> Ready for Hybrid Multicloud environments with on-premises and off-premises options in 
IBM Cloud and other partners 
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3.1.1 IBM Power Systems H922 (9223-22S) and H924 (9223-42S) running SAP 


The Power H922 and H924 for SAP HANA server supports two processor sockets and up to 
4 TB of memory. These servers support a mix of classic SAP R/3 and S4HANA coexisting in 
the same server. Linux is required as the primary operating system with a minimum of 75% of 
total cores activated. AIX and IBM i also can be chosen as secondary operating systems, with 
a maximum of 25% of total cores activated across both. 


Currently, these servers support only HANA version 2.0 in little-endian mode. 


The following Linux offerings are supported for production SAP workloads: 
> Red Hat Linux for SAP and Red Hat Linux 8: Running HANA 2.0 SPS04 and newer 


>» SUSE SLES for SAP, SUSE SLES 12 SP3, or SUSE SLES 15: Running HANA 2.0 SPSO3 
and newer 


3.1.2 Virtual Persistent Memory (vPMEM) 


A new capability in the PowerVM platform of IBM Power Systems, Virtual Persistent Memory 
(vPMEM) is designed to create persistent memory volumes by using the existing DRAM 
technology. 


By maintaining data persistence across application and partition restarts, it is engineered to 
allow customers to use fast restarts of a workload by using persistent memory for most of 
their planned maintenance and unplanned outages without compromising the performance of 
HANA during normal use. 


vPMEM enables the restart of SAP HANA environments approximately 17x faster based on 
IBM tests, in case of any planned or unplanned maintenance work like, such as HANA-related 
patching, upgrades, operating system-related patching, upgrades, or unplanned outages that 
are related to HANA or the operating system. 


3.2 Cloud-ready H922 and H924 servers 


Cloud capabilities are a prerequisite for the use of enterprise-level IT resources. A rich set of 
technologies is available around Power System H922 (9223-22S) and Power System H924 
(9223-42S. These technologies help modernize services with the strategic cloud initiatives of 
your business. 


The following offerings are most important: 


IBM PowerVC 

IBM Power Systems Virtual Servers 

Red Hat OpenShift Container Platform for Power 
IBM Cloud Pak® for Multicloud Management 


vvvy 


Chapter 3. SAP HANA and Cloud Private 127 


3.2.1 IBM PowerVC 


IBM PowerVC is an advanced virtualization and cloud management offering that is built on 
OpenStack. It provides simplified virtualization management and cloud deployments for IBM 
AIX, IBM i, and Linux virtual machines (VMs) that are running on IBM Power Systems. 
PowerVC is designed to improve administrator productivity and simplify the cloud 
management of VMs on Power Systems servers. 


It also provides a multi-tenant Infrastructure-as-a-Service (laaS) layer in the cloud software 
stack with which administrators can quickly provision new VMs in minutes. It also provides 
numerous operational benefits, including the following examples: 


One-click system evacuation for simplified server maintenance 

Dynamic resource optimization (DRO) to balance server usage during peak times 
Automated VM restart to recover from failures 

Importing and exporting VM images for cloud mobility 
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It also enables DevOps capabilities, such as “infrastructure as code” by way of HashiCorp 
Terraform. Terraform can provision POWER resources through PowerVC by using the 
included OpenStack provider. PowerVC provides the foundational technology on top of which 
the rest of the POWER cloud stack builds. 


PowerVC provides integration to higher-level cloud orchestrators that are based on 
OpenStack technology. It helps Power Systems customers lower their total cost of ownership 
with a simplified user experience that enables simple cloud deployment and movement of 
workloads and policies to maximize resource usage. 


3.2.2 IBM Power Systems Virtual Servers 


IBM Power Systems Virtual Servers on IBM Cloud is an laaS offering to deploy a Power 
virtual server, also known as a logical partition (LPAR), in a matter of minutes. Environments 
that typically relied on an on-premises only infrastructure can extend Power Systems 
resources into the cloud. The use of IBM Power Virtual Servers on IBM Cloud is an alternative 
to avoid the large capital expense or added risk when moving workloads to a public cloud. 


An IBM Cloud integrates IBM AIX, IBM i, or Linux (bringing your own licenses model) 
capabilities into the IBM Cloud experience. This integration results in fast, self-service 
provisioning, and flexible management on-premises and off. It also enables access to a stack 
of enterprise IBM Cloud services with pay-as-you-use billing that allows scale up and out. 


An easier and faster deployment of an IBM Power Systems Virtual Servers on IBM Cloud 
instance meets any business need. It also opens an opportunity to create a hybrid cloud 
environment that facilitates easy control workload demands. 


Recently, IBM announced the general availability of IBM Power Systems Virtual Server for 
SAP HANA and NetWeaver platform (ECC) workloads. SAP’s own performance tools and 
published benchmarks show that IBM runs the most-powerful SAP-certified laaS. 
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SAP customers can now take advantage of the flexibility of IBM’s SAP-certified Infrastructure 
as a Service Power Virtual Server environment to accelerate their migration to SAP 
S/4HANA. The underpinnings of these virtual servers are the industry-leading IBM Power 
Systems E980 servers with each virtual server capable of scaling up to 143 cores and 15 TB 
of RAM for the largest and most demanding SAP HANA workloads. 


This announcement further expands the capabilities of the Power Systems Virtual Server 
offering to deliver a resilient and secure platform for business for enterprise-scale SAP 
deployments. IBM is announcing certification of the SAP NetWeaver application stack for 
production use on the IBM Power Systems Virtual Server offering, which rounds out the 
best-in-class SAP offering. 


For more information, see IBM Power Systems Virtual Servers - Getting started tutorial. 


IBM Power Systems Virtual Server Sales and Distribution benchmark: SAP published 
on June 8, 2020 that a Power E980 on Power Virtual Server performs 907,820 SAPs. The 
full report of this benchmark is available for download here. 


3.2.3 Red Hat OpenShift Container Platform for Power 


Red Hat OpenShift Container Platform for Power (5639-OCP) provides a secure, 
enterprise-grade platform for on-premises, private platform-as-a-service clouds on Power 
Systems servers. It brings together industry-leading container orchestration from Kubernetes, 
advanced application build and delivery automation, and Red Hat Enterprise Linux certified 
containers for Power Systems. 


It also brings developers and IT operations together with a common platform. It provides 
applications, platforms, and services for creating and delivering cloud-native applications and 
management so IT can ensure that the environment is secure and available. 


Red Hat OpenShift Container Platform for Power provides enterprises the same functions as 
the Red Hat OpenShift Container Platform offering on other platforms. Key features include 
the following examples: 


> Aself-service environment for application and development teams. 


> Pluggable architecture that supports a choice of container run times, networking, storage, 
Continuous Integration-Continuous Deployment (CI-CD), and more. 


> Ability to automate routine tasks for application teams. 


Red Hat OpenShift Container Platform subscriptions are offered in two-core increments that 
run in a virtual guest. The purchase of a separate Red Hat Enterprise Linux subscription 
(5639-RLE) is required as the base operating system. 
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3.2.4 IBM Cloud Pak for Multicloud Management 
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IBM Cloud Pak for Multicloud Management, running on Red Hat OpenShift, provides 
consistent visibility, governance, and automation across the entire hybrid multicloud 
landscape. Offered as part of this Cloud Pak are the following critically important 
management applications to hybrid multicloud: 


> IBM Cloud Automation Manager (CAM) 


This application provides advanced multicloud management capabilities. By using 
HashiCorp Terraform as its underlying engine, CAM enables connectivity to numerous 
cloud infrastructures, including PowerVC (OpenStack), IBM Cloud, AWS, Azure, and 
Google. 


CAM can provision VMs (including LPARs by way of PowerVC) and containers so that 
users can create software catalog entries that build complex multitier applications with a 
single click. Because CAM is delivered as part of an IBM Cloud Pak, it inherently runs on 
Red Hat OpenShift and creates a centralized management plane from which you can 
deploy all of your application 


> IBM Multicloud Manager 


This application provides a single multicloud dashboard that enables organizations to 
oversee multiple cloud endpoints wherever they are on the public or private cloud. IBM 
Multicloud Manager provides consistent visibility, governance, and automation across the 
hybrid multicloud. 
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Related publications 


The publications that are listed in this section are considered particularly suitable for a more 
detailed discussion of the topics that are covered in this paper. 


IBM Redbooks 


The following IBM Redbooks publications provide more information about the topic in this 
document. Note that some publications that are referenced in this list might be available in 
softcopy only: 


> 


> 


> 


> 


IBM Power Systems H922 and H924 Technical Overview and Introduction, REDP-5498 


IBM Power Systems S922, S914, and S924Technical Overview and Introduction 
Featuring PCle Gen 4 Technology, REDP-5595 


IBM Power System AC922 Technical Overview and Introduction, REDP-5494 

IBM Power System L922 Technical Overview and Introduction, REDP-5496 

IBM PowerAl: Deep Learning Unleashed on IBM Power Systems Servers, SG24-8409 
IBM PowerVM Best Practices, SG24-8062 

IBM PowerVM Virtualization Introduction and Configuration, SG24-7940 

IBM PowerVM Virtualization Managing and Monitoring, SG24-7590 


You can search for, view, download or order these documents and other Redbooks, 
Redpapers, Web Docs, draft and additional materials, at the following website: 


ibm. com/redbooks 


Online resources 


The following websites are also relevant as further information sources: 


> 


IBM Fix Central: 

http://www.ibm.com/support/fixcentral/ 

IBM Knowledge Center: 

http://www.ibm.com/support/knowledgecenter/ 

IBM Knowledge Center: POWER systems information: 

https://www. ibm. com/support/know]edgecenter/POWER9/ 

SAP HANA on IBM Power Systems and IBM System Storage Techdoc: 
http://www-03.ibm.com/support/techdocs/atsmastr.nsf/webindex/wp102502 


IBM Knowledge Center: Migration combinations of processor compatibility modes for 
active Partition Mobility: 


https: //www. ibm. com/support/know]ledgecenter/9223-42S/p9hc3/p9hc3_pcmcombosact.h 
tm 
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> IBM SAP HANA Servers for Infrastructure Solutions: 
https: //www.ibm.com/it-infrastructure/power/sap-hana 
> IBM Portal for OQpenPOWER - POWER9 Monza Module: 


https://www. ibm. com/systems/power/openpower/tgcmDocumentRepository.xhtml ?aliasI 
d=POWER9_Monza 


> IBM Power Systems: 
https://www.ibm.com/it-infrastructure/power 

> IBM Storage: 
https://www.ibm.com/it-infrastructure/storage 

> IBM Systems Energy Estimator: 
http://www-912.ibm.com/see/EnergyEstimator/ 

> IBM Systems Workload Estimator: 
https://wle.mybluemix.net/wle/EstimatorServlet 

> IBM System Planning Tool: 


https: //www. ibm. com/support/pages/ibm-system-p]anning-tool -power-processor-base 
d-systems-0 


> NVIDIA Tesla V100: 
https: //www.nvidia.com/en-us/data-center/tesla-v100/ 
> OpenCAPI: 
http://opencapi.org/technical/use-cases/ 
> OpenPOWER Foundation: 
https: //openpowerfoundation.org/ 
> Power Systems Capacity on Demand website: 
https: //www.ibm.com/support/pages/node/665179 
> IBM Power Systems Performance and capacity: 
https: //www. ibm. com/support/pages/node/668145 
> Nigel Griffiths’ IBM Power System VLOG: 
https: //www. youtube. com/nigelargriffiths 


Help from IBM 


IBM Support and downloads 


ibm.com/support 


IBM Global Services 


ibm.com/services 
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